You need a chat server program which can persist conversations (like have a DB or pesistent store at the server side). Chat client(s) can connect to chat server and communicate with each other.
Google for chat server programs in
java, you can find many.