• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Multi Files Transfer from client machines to server machines using socket communication in java

 
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

if any one have the code to transfer files from client to server , Could you please [share] the code for multi file transfering from client to server.
Expecting your valuable response ..

My requirement is transfering all files from particular folder in client machine to target folder in server machine, I have the code but its transferring only one file to server machine after that connection disconnect. plese guide me to complete my requirement. (Java Socket Communication)

Thanks in Advance..
 
author
Posts: 23951
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
https://coderanch.com/how-to/java/UseTheForumNotEmail
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Henry Wong wrote:https://coderanch.com/how-to/java/UseTheForumNotEmail

Sorry for that. I was looking in thread but i didnt get a solution for Multi file transfer using socket communication in java, could you please guide , My code is transferring only one file at a time..


I am new to this concept. for file transfering which is the best mechanism in java , either Socket communication or FTP?
please advice me the best concept to do the file transfer mechanism .

Expecting your valuable reply.
Thanks in Advance .
 
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:

Henry Wong wrote:https://coderanch.com/how-to/java/UseTheForumNotEmail

Sorry for that. I was looking in thread but i didnt get a solution for Multi file transfer using socket communication in java, could you please guide , My code is transferring only one file at a time..



If you know how to do one file, then you know how to do multiple files.

I am new to this concept. for file transfering which is the best mechanism in java , either Socket communication or FTP?



There is no single "best" mechanism. The best one for you in this particular situation is the one that best meets your requirements in this particular situation.

 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jeff Verdegan wrote:

Riyas Hameed wrote:

Henry Wong wrote:https://coderanch.com/how-to/java/UseTheForumNotEmail

Sorry for that. I was looking in thread but i didnt get a solution for Multi file transfer using socket communication in java, could you please guide , My code is transferring only one file at a time..



If you know how to do one file, then you know how to do multiple files.

I am new to this concept. for file transfering which is the best mechanism in java , either Socket communication or FTP?



There is no single "best" mechanism. The best one for you in this particular situation is the one that best meets your requirements in this particular situation.



Thanks for your reply, but I am getting connection refused after one file transferred to server, I am looking on that.

Here My code is :

Client part :

Server part:

kindly advise me , what mistake i did in my code ..
 
Jeff Verdegan
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
In the future, please UseCodeTags so your code will be readable.


You're closing your socket's OutputStream after each file. You need to either:

1) Keep the socket and its streams open throughout the entire loop over all files.

OR

2) Create a new socket connection for each file, and close it in a finally block at the end of transferring that file.

I'd recommend #2. If you go with #1, you'll have to develop an additional protocol to delineate the files. That's not necessarily difficult, but since you're having these difficulties so far, it's probably best to start simple.
 
Sheriff
Posts: 22781
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:


0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks a lot to all, I will do the same , Iam new to this forum going forward i will ad my code in code tag. I will do what you have mentioned and come back to you if i am facing any issue.
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sir ,Please see the below code .
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks a lot , As per your words, I did the changes , Its working fine..I jus added one more while loop in server part







Now its working fine, This client application will run more than 500 Machines.So how can i make the best perofrmance .please suggest me to do this part. Thanks in Advacne.
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:Thanks a lot , As per your words, I did the changes , Its working fine..I jus added one more while loop in server part.

Now its working fine, This client application will run more than 500 Machines.So how can i make the best perofrmance .please suggest me to do this part. Thanks in Advacne.

 
Jeff Verdegan
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:Now its working fine, This client application will run more than 500 Machines.So how can i make the best perofrmance .please suggest me to do this part. Thanks in Advacne.



For parallel I/O such as this, the two main general approaches to optimize performance are 1) buffering, and 2) multithreading. You're already doing #1, although you may find that tweaking your buffer size up or down helps somewhat, so the main thing now is to use mulitple threads.

If I understand your scenario correctly you have many clients receiving files from a server and storing them on their local file systems. So you'll want multithreading in two places. 1) On the server, you'll want multiple threads to handle clients' requests. You'll have to experiment to find the optimal number. It could be anywhere between a few threads and a few hundred. 2) On the client, you might want two threads--one for reading from the server and one for writing to the file. Or that might not help at all. Impossible to say without trying it in the specific situation in which it will be used.

Other than that, the only thing you can do is see if the app is meeting your performance requirements, and if not, use a profiler to find bottlenecks.
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jeff Verdegan wrote:

Riyas Hameed wrote:Now its working fine, This client application will run more than 500 Machines.So how can i make the best perofrmance .please suggest me to do this part. Thanks in Advacne.



For parallel I/O such as this, the two main general approaches to optimize performance are 1) buffering, and 2) multithreading. You're already doing #1, although you may find that tweaking your buffer size up or down helps somewhat, so the main thing now is to use mulitple threads.

If I understand your scenario correctly you have many clients receiving files from a server and storing them on their local file systems. So you'll want multithreading in two places. 1) On the server, you'll want multiple threads to handle clients' requests. You'll have to experiment to find the optimal number. It could be anywhere between a few threads and a few hundred. 2) On the client, you might want two threads--one for reading from the server and one for writing to the file. Or that might not help at all. Impossible to say without trying it in the specific situation in which it will be used.

Other than that, the only thing you can do is see if the app is meeting your performance requirements, and if not, use a profiler to find bottlenecks.




Thank a lot for your support , In my requirement , All clients will transfer the files to the centralized server and save in server location.

So I have to create two threads in server side one for reading and one for writing ? please correct me If i am wrong. I have implemented thread concept in server side , some time some of the bytes are missing while writing in server side , now i am facing the issue . can i share my code here ??
 
Jeff Verdegan
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:
Thank a lot for your support , In my requirement , All clients will transfer the files to the centralized server and save in server location.

So I have to create two threads in server side one for reading and one for writing ? please correct me If i am wrong. I have implemented thread concept in server side



If all 500 clients are sending their files at the same time, I would start with something like 10-20 threads reading from clients, putting into queues (1 queue per destination file) and 1-5 threads reading from the queues and writing to files. I'd probably use ThreadPoolExecutors and ArrayBlockingQueues or LinkedBlockingQueues.

Since I don't know your system or your requirements at all, this is just starting point. You'll have to do some research into the relevant classes, and do some testing and tweaking on your setup to see what works best.

, some time some of the bytes are missing while writing in server side , now i am facing the issue



Then there's a bug in your code.

. can i share my code here ??



Probably the whole thing will be too much for anybody to want to read. You'll have better luck if you can create an SSCCE that shows the relevant parts of your code and ideally demonstrates the problem, without any extra stuff that's not directly related to the problem.
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jeff Verdegan wrote:

Riyas Hameed wrote:
Thank a lot for your support , In my requirement , All clients will transfer the files to the centralized server and save in server location.

So I have to create two threads in server side one for reading and one for writing ? please correct me If i am wrong. I have implemented thread concept in server side



If all 500 clients are sending their files at the same time, I would start with something like 10-20 threads reading from clients, putting into queues (1 queue per destination file) and 1-5 threads reading from the queues and writing to files. I'd probably use ThreadPoolExecutors and ArrayBlockingQueues or LinkedBlockingQueues.

Since I don't know your system or your requirements at all, this is just starting point. You'll have to do some research into the relevant classes, and do some testing and tweaking on your setup to see what works best.

, some time some of the bytes are missing while writing in server side , now i am facing the issue



Then there's a bug in your code.

. can i share my code here ??



Probably the whole thing will be too much for anybody to want to read. You'll have better luck if you can create an SSCCE that shows the relevant parts of your code and ideally demonstrates the problem, without any extra stuff that's not directly related to the problem.




Thank you very much, yes I have to go through all other thing which you have mentioned in the previous message . I am not sure but atleast minimum 50 to 100 Machine will send a file to server at a time . i will discuss with them and come back to you ..
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:

Jeff Verdegan wrote:

Riyas Hameed wrote: Thank a lot for your support , In my requirement , All clients will transfer the files to the centralized server and save in server location. So I have to create two threads in server side one for reading and one for writing ? please correct me If i am wrong. I have implemented thread concept in server side

If all 500 clients are sending their files at the same time, I would start with something like 10-20 threads reading from clients, putting into queues (1 queue per destination file) and 1-5 threads reading from the queues and writing to files. I'd probably use ThreadPoolExecutors and ArrayBlockingQueues or LinkedBlockingQueues. Since I don't know your system or your requirements at all, this is just starting point. You'll have to do some research into the relevant classes, and do some testing and tweaking on your setup to see what works best.

, some time some of the bytes are missing while writing in server side , now i am facing the issue

Then there's a bug in your code.

. can i share my code here ??

Probably the whole thing will be too much for anybody to want to read. You'll have better luck if you can create an SSCCE that shows the relevant parts of your code and ideally demonstrates the problem, without any extra stuff that's not directly related to the problem.

Thank you very much, yes I have to go through all other thing which you have mentioned in the previous message . I am not sure but atleast minimum 50 to 100 Machine will send a file to server at a time . i will discuss with them and come back to you ..




Here is my code for read and write in file :

Server side : read the data from the file which have sent from client and writ in a file .


Client code :




Some of the file bytes are missing while writing to server . please advise me to solve this . Thanks in Advance
 
Marshal
Posts: 28177
95
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you read the documentation for the "read" method you're using, you will see that it returns -1 when it gets to the end of the stream. But you aren't checking for that. You are also considering 0 as a reason to stop reading.

But when the "read" method returns 0, that just means it has no bytes for you just at the moment. It doesn't mean that there will never be any more bytes. So fix your end-of-stream tests.
 
Jeff Verdegan
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Paul Clapham wrote:If you read the documentation for the "read" method you're using, you will see that it returns -1 when it gets to the end of the stream. But you aren't checking for that. You are also considering 0 as a reason to stop reading.

But when the "read" method returns 0, that just means it has no bytes for you just at the moment. It doesn't mean that there will never be any more bytes. So fix your end-of-stream tests.



This was already mentioned by Rob Spoor many replies and 4 days ago. Perhaps this time it will sink in. :-)
 
Paul Clapham
Marshal
Posts: 28177
95
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You're right. And not only that, Rob described exactly what code changes should be done. And that was the only thing in his post. So yeah, there was really no excuse for
not noticing his post.
 
Rob Spoor
Sheriff
Posts: 22781
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sometimes people are focusing so much on what one person said that what others say doesn't sink in. This wouldn't be the first time when a post of mine had to be brought under the attention again because it was overlooked at first.
 
Jeff Verdegan
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think it's happened to us all a number of times. You get in a thread with multiple respondents and multiple issues to be addressed and the OP gets overwhelmed.
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Rob Spoor wrote:

Riyas Hameed wrote:


0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.






Rob, I tried already what you said in the above conversation, even though i got the same error only, extremely sorry, forget to noticed that one.

 
Jeff Verdegan
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:

Rob Spoor wrote:

Riyas Hameed wrote:


0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.




Rob, I tried already what you said in the above conversation, even though i got the same error only, extremely sorry, forget to noticed that one.



The last code you posted shows:


Which is still the same mistake you were making in the beginning, and does not incorporate Rob's (and Paul's) advice. Fix that, and then if you're still having trouble, post your latest code, or an SSCCE that accurately represents it.
 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:

Rob Spoor wrote:

Riyas Hameed wrote:


0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.






Rob, I tried already what you said in the above conversation, even though i got the same error only, extremely sorry, forget to noticed that one.



 
Riyas Hameed
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Riyas Hameed wrote:

Riyas Hameed wrote:

Rob Spoor wrote:

Riyas Hameed wrote:


0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.






Rob, I tried already what you said in the above conversation, even though i got the same error only, extremely sorry, forget to noticed that one.





Thanks a lot for your support, Now it is working fine . I have changed the condition to till -1. Thanks for your time ..
reply
    Bookmark Topic Watch Topic
  • New Topic