Win a copy of The Business Blockchain this week in the Cloud forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

File transfer using sockets and streams

 
Mousa A. Rafi
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is a simple client server program. The client requests for a file to the server and if the file is present on the server server sends it to the client.
I've tried it with sockets and streams, but it's not working properly. Most of the time some data loss occurs and seems very slow. I assume that as the server sends data byte by byte and the client reads the same way and since they are separate programs, the proper sequencing of sending and receiving is not happening. So, is it actually the problem or is there anything else? How to solve it (in the best way)?

Here's my server code:


Here's my client code:

[ May 05, 2005: Message edited by: Mousa A. Rafi ]
 
Joe Ess
Bartender
Posts: 9337
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Readers are only for character data. If you use a reader on binary data (i.e. an image file) the data is truncated to match the current character set.
 
Mousa A. Rafi
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Actually, if you look closely to my code, you'll find that the part that deals with sending and receiving a file does not use any reader. I used BufferedInputStream and BufferedOutputStream to do the job. I used readers (BufferedReader)only to read messages sent from the client to server and vice versa. Besides the program behaves inappropriately even when I'm dealing with text files.
 
Joe Ess
Bartender
Posts: 9337
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I apologize for that. It's a knee-jerk reaction every time I see readers and sockets in the same code.
Your code works fine for me. No data loss. One reason it's slow is that flush() that you call on line 79 of the server. You only need to do that once, before invoking close(). It's also a good idea to flush the file output stream on the client just to be sure everything gets written.
Moving data byte-by-byte is time consuming. Check out the IO Chapter of JavaTM Platform Performance: Strategies and Tactics for the best way to use buffering to streamline throughput.
If you are still experiencing data loss, give us a test case that fails. I tested it with various images between 14 and 140k and had no problems.
 
Mousa A. Rafi
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Joe Ess, thanks for trying it out. I'll try it with the changes you've mentioned. However, you might want to try it out with some text files. I tested it with few html and java files. Most of the time some characters were lost at the beginning of the files. This happens on Windows. Originally I coded and tested on Linux where thing were worse. Sometimes the text files were totally blank.
 
James Carman
Ranch Hand
Posts: 580
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That's why FTP has both binary and text mode. In text mode, you can make sure you write and read to and from the stream using the same encoding (UTF-8 for example). You read and write from and to disk using the default encoding of the OS. For binary data, you can just transfer it byte for byte. There's also the problem with mismatched line terminators.
[ May 06, 2005: Message edited by: James Carman ]
 
Mousa A. Rafi
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
James Carmen, what you are saying is true. But, shouldn't we get the exact copy of the file when using binary mode? We might not have a carriage return at the end but atleast we shouldn't have missed a character.

In the previous post what I meant is I tried the server and the client program on the same machine. Once on Linux and once on Windows (for example). It's not like my server is on Linux and the client is on Windows or vice versa. So, there actually shouldn't be any hassle of appending any extra character at the first place.
 
Joe Ess
Bartender
Posts: 9337
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by James Carman:
In text mode, you can make sure you write and read to and from the stream using the same encoding (UTF-8 for example).


One can move text files as binary. Just copying bytes won't do any translation on the data. One can't treat binary files as text (i.e. using a Reader in Java as I mentioned earlier). It would also be a Bad Idea to us a Reader with a charset on a file of a different encoding.
As for testing, Mousa, I tried a java source file (523 bytes) and an HTML file (9k) and the copy is identical to the original. Are you certain that you have recompiled your source and you don't have any old versions in the classpath?
 
Joe Ess
Bartender
Posts: 9337
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Mousa A. Rafi:
shouldn't we get the exact copy of the file when using binary mode? We might not have a carriage return at the end but atleast we shouldn't have missed a character.


To belabor the point, a carrage return is a character, so they would get copied as well in a binary copy. Unix and Windows recognize different line seperators, Unix "\n", Windows "\n\r" (or vice versa, can't remember), and that causes some confusion to some folks the first time they move files between the two.
 
Mousa A. Rafi
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sorry Joe, but the problem still exists. I've tried with the modifications you suggested. It now works lot faster. I could transfer a mp3 file of 5MB in seconds which would take minutes before and couldn't find any transfer problem with that file.

Changed code on the server:


Actually the problem don't happen everytime. In my case, the last time I tried the programs: first I transferred a html file, then a java file and then another java file (all located in the same root directory). Then I checked the transferred files. The first two were ok, but the last java file was totally empty.


Are you certain that you have recompiled your source and you don't have any old versions in the classpath?

I've recompiled the source codes and as a result it works faster. I'm certain that's not the problem.
 
Joe Ess
Bartender
Posts: 9337
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Mousa A. Rafi:

Actually the problem don't happen everytime. In my case, the last time I tried the programs: first I transferred a html file, then a java file and then another java file (all located in the same root directory). Then I checked the transferred files. The first two were ok, but the last java file was totally empty.

I tried it again. The only problem I got was when the file already existed on the client side. The server side crashes.
How are you checking the files? Are you opening them in an editor, doing a command-line "dir" or using the file explorer? File explorer doesn't automatically update, so the file size could be way off from what the file really is.

Originally posted by Mousa A. Rafi:

I've recompiled the source codes and as a result it works faster. I'm certain that's not the problem.

Just trying every angle I can think of. . .
 
Mousa A. Rafi
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That's interesting. I didn't handle the situation when the file already exists on client side. But, the server never generated an error. I wonder why.

I checked the files with notepad on Windows and gedit on Linux.
 
Joe Ess
Bartender
Posts: 9337
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Be aware that notepad does not handle UNIX-style line seperators (i.e. '\n'). Your file will look mangled but it's actually notepad being stupid. This may be the cause of some of your problems. Use Wordpad or a programmer's text editor that handles line seperators properly.
 
Sadrul Chowdhury
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello.

The problem occurs in the server, when you use PrintWriter (out) and BufferedOutputStream (fout) on the same output-stream. As it appears, the combination of these two is the course of the problem. flush()ing after sending "Yes" would sanitize everything and should solve the issue.

-- Sadrul Habib Chowdhury
 
Mousa A. Rafi
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Joe Ess:

I tried it again. The only problem I got was when the file already existed on the client side. The server side crashes.



Joe Ess, what version of JDK or any IDE are you using? The problem you mentioned never occured to me. So, I can't tress why it crashes on your machine. But, it probably is the main problem, because after handling it, the applications now work fine.
 
Joe Ess
Bartender
Posts: 9337
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator


Here's the console trace of the server. This is your original code. I changed some of the file/dir names. I download the file once, then the server crashes on the second download.

Now that I'm testing it again, I see the problem doesn't happen for small files (a 26k file works fine). bzLifecycle.png is 63k. I think what happens is the server is writing the file to the socket and the client, who sees the file exists, closes the socket. Hence the "Connection reset by peer" message. With a small file, the server completes writing before the client can close the socket. A TCP/IP packet will hold several k depending on your network settings.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic