I came across some information that JMS can be used to create "message queues", is it?
Yes, JMS is the standard Java API for message queues, like ActiveMQ and OpenMQ.
Who does initiate the notification in a user-server application? Does server send the notifications to the client without client's request or the client requests for the notifications (like an ajax call is made automatically from client side as the client signs into his account)? How does it usually happen technically?
Both are possible. The client can pull (with an AJAX call) or the server can push (using server-sent events or web sockets).
I'm confused about JMS. I want to repeat my question, Is there no involvement of JMS in sending email, SMS, have chat with user and send "notification" to user's profile?
It is conceptually possible that an SMS/email/chat implementation uses message queues (and thus JMS) in the background, but that's an implementation detail that has nothing to do with SMS/email/chat. Please get the notion out of your head that the "message" in "message queueing" has anything to do with the "message" in SMS/email/chat.