• Post Reply Bookmark Topic Watch Topic
  • New Topic

Loading large amounts of data from separate JVM  RSS feed

 
Alfred C. Thomas
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all

I am developing an application that needs to import large amounts of data from a flat file, and make it available to separate applications on separate JVM/accross a network. When the data changes, a JMS message is sent to the client, which then initiates the data transfer.
What I need now is the most efficient way of sending large amounts of data (Some of the files are 1GB) accross to the client. Can anyone please help me with some ideas on achieving this?

Regards
Alfred
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The simplest thing would be to start an FTP process, thats a well understood protocol. The disadvantage would be that it is linear and only one Thread would be doing the transfer so it would be slower than fancy multithread techniques.
The Apache Commons project has an FTP library.
Bill
 
Alfred C. Thomas
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Bill

Thanks for the response. I don't actually want to send the file across. I want the a single application doing the import process from file. The client applications would then request the already imported data from the "server". The reason for the application is to have a generic import facility, to take the complexity of parsing the binary files away from the client.
The question is this, what is the best way of making these large amounts of data objects available to the clients. One way I am investigating is to send the data to a JMS queue, and have a message driven bean persist them on the other side. But I am not sure whether this is the right approach (at all).

Regards,
Alfred
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!