• Post Reply Bookmark Topic Watch Topic
  • New Topic

Sockets, threads and JDBC - design question  RSS feed

 
Archanaa Panda
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,

I am currently in the architecture analysis and design phase for a project which may very likely involve pure socket, thread and JDBC programming.

In this project, thousands of client devices may periodically send measurement data that has to be stored into a database system. The communication with the client devices would be over TCP/IP sockets.

These sockets would probably be long-lived because the server may have to collect information from the client devices in particular scenarios.

This data has to be decompressed/transformed into a format for storing into the database.

As bulk amounts of data are expected, this solution has to be highly performant and scalable.

I have a few questions here-
1. For each socket opened between the client and server, would it be advisable to dedicate a thread for reading the socket? Is there any limit to the threads and sockets that can be kept open on the server? If yes, it would probably be better to reuse the threads across various open sockets. How long can these sockets be kept alive?

2. Each request's data has to be processed (probably transformed) and inserted into database tables. Is it better to acquire a connection from a connection pool for each thread and insert the data into the database table or would a better architecture be to dedicate a thread for batch insertion of database tables over a single connection and make the receiver threads store data in a queue wherefrom this data would be picked up by this thread and inserted into the database in bulk. Again, how many threads for database interaction are recommended.

3. If I need to test this out, what would be the right approach/tools to use?


Thanks and Regards,
Archanaa
 
Jeff Albertson
Ranch Hand
Posts: 1780
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1. Sockets + Server => SocketChannels. http://java.sun.com/developer/JDCTechTips/2003/tt0909.html#1

Also, you may be able to use a more common pattern, like that of a web-service,
and instead of having "long" lived connections, have the server return to
the client a "session token" that can be used to identify further calls
as continuations of the conversation.

2. Connection Pool.

Of course, your server could well be running in a container that provides
connection pooling so you'd get that "for free".
[ December 14, 2005: Message edited by: Jeff Albrechtsen ]
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Here's some code from a web server that does a similar mission in life:

For each inbound request it creates a new runnable handler and runs it on a thread from a pool. This particular pool will grow to as many threads as it needs. You could make one that has a fixed size and queues requests up when too many come in at once. Does that fit the problem?
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!