Copying Files over a Socket

[J Mangual]

This may be more of an I/O question...

I want to copy files over a network reliably. The program I have works except for two things:

The socket connection should terminate after receiving the file. In runtime, the connection does not terminate. My guess is, the client is stuck in the for loop that reads the file from the network.

When I used a test file, numbering the lines from 1 to 1000 (each lines reads "[number] ----- L" ), I noticed about 60 extra lines at the end of the file. Probably this is because I read into and copy from 1024-byte arrays which don't totally get written over at the end of the file.

So, how do I detect the end of file when I am using byte arrays like this? Or is there an even bigger issue.

import java.net.*;
import java.io.*;
public class FileClientW{
public static void main (String [] args ) throws IOException {
// localhost for testing
Socket sock = new Socket("127.0.0.1",4444);
System.out.println("Connecting...");
// receive file
InputStream in = sock.getInputStream();
FileOutputStream out = new FileOutputStream(args[0]);
byte[] mb = new byte[1024];
for(int c = in.read(mb, 0, 1024); c > 0; c = in.read(mb, 0, 1024)){
out.write(mb, 0, 1024);
}
out.close();
sock.close();
}
}

import java.net.*;
import java.io.*;
public class FileServerW {
public static void main (String [] args ) throws IOException {
// create socket
ServerSocket servsock = new ServerSocket(4444);
while (true) {
System.out.println("Waiting...");
Socket sock = servsock.accept();
System.out.println("Accepted connection : " + sock);
// sendfile
File myFile = new File (args[0]);
FileInputStream in = new FileInputStream(myFile);
OutputStream out = sock.getOutputStream();
System.out.println("Sending...");
byte[] mb = new byte[1024];
for(int c = in.read(mb, 0, 1024); c > -1; c= in.read(mb, 0 ,1024)){
out.write(mb, 0, 1024);
}
//sock.close();
}
}
}

[Joe Ess]

 

J Mangual wrote:
When I used a test file, numbering the lines from 1 to 1000 (each lines reads "[number] ----- L" ), I noticed about 60 extra lines at the end of the file. Probably this is because I read into and copy from 1024-byte arrays which don't totally get written over at the end of the file.

Have a look at our FAQ: ReadDoesntDoWhatYouThinkItDoes, though in this case, write() doesn't do what you think it does. You always write 1024 bytes no matter how many bytes you have read.
Also, I believe if you do not close the output stream of the socket, EOF is never sent, so your client will just block waiting for input. Why did you comment out the socket.close()? Try flushing and closing the output stream.

[J Mangual]

Is the FAQ suggesting we are supposed to load the entire file into memory when we receive it?

Here's a quote from Eliotte Rusty Harold's "Java I/O":

Traditional synchronous I/O is designed for traditional applications...

Files may be large but not huge. It's possible to read an entire file into memory.

An application reads or write only to a few files or network connections at the same time, ideally using only one stream at a time

The application is sequential. It won't be able to do muc until it's finished reading or writing a file.

So in the FAQ it looks like they read the whole file into the buffer and then process it. Do I need to read the whole file into memory just to copy it accurately?

A correct usage of read() would be to read chunks of a file and collect them in a buffer like so:

 ByteArrayOutputStream buffer = new ByteArrayOutputStream();
File aFile = new File("input.data");
InputStream is = new FileInputStream(aFile);
byte[] temp = new byte[1024];
int read;
while((read = is.read(temp)) > 0){
buffer.write(temp, 0, read);
}
byte[] data = buffer.toByteArray();
// process the data array . . . 

[Joe Ess]

 

J Mangual wrote:Is the FAQ suggesting we are supposed to load the entire file into memory when we receive it?

The FAQ is suggesting that you write only the data you receive. Note the use of the variable "read":

while((read = is.read(temp)) > 0){
buffer.write(temp, 0, read);
} 

As the FAQ says, the call to read() may not fill the buffer temp. Your code always writes 1024 bytes. The code in the FAQ will only write as many bytes as is read.
It just so happens that in the example used in the FAQ, the file is to be read into memory fully, then processed.

[J Mangual]

I changed it so that it reads a variable number of bytes and there is no extra goop at the end of the file now. I try to read and write to the buffer as much as possible, but I realize there is no guarantee to how many bytes are actually written.

The reason I don't want to close the output stream is because I may want to send several files over the same connection. So it's important I find some way of getting the end of file, but maybe that's for another thread. Thanks.

Client:

import java.net.*;
import java.io.*;
public class FileClientW{
public static void main (String [] args ) throws IOException {
// localhost for testing
Socket sock = new Socket("127.0.0.1",4444);
System.out.println("Connecting...");
// receive file
InputStream in = sock.getInputStream();
FileOutputStream out = new FileOutputStream(args[0]);
byte[] mb = new byte[1024];
for(int c = in.read(mb, 0, 1024); c > 0; c = in.read(mb, 0, 1024)){
out.write(mb, 0, c);
}
out.close();
sock.close();
}
}

Server:

import java.net.*;
import java.io.*;
public class FileServerW {
public static void main(String[] args) throws IOException {
// create socket
ServerSocket servsock = new ServerSocket(4444);
while (true) {
System.out.println("Waiting...");
Socket sock = servsock.accept();
System.out.println("Accepted connection : " + sock);
// sendfile
File myFile = new File(args[0]);
FileInputStream in = new FileInputStream(myFile);
OutputStream out = sock.getOutputStream();
System.out.println("Sending...");
byte[] mb = new byte[1024];
for(int c = in.read(mb); c > -1; c= in.read(mb)){
out.write(mb, 0, c); }
}
}
}

[Joe Ess]

 

J Mangual wrote: I try to read and write to the buffer as much as possible, but I realize there is no guarantee to how many bytes are actually written.

Try searching the forum. I'm certain we've discussed that topic more than once.