How to read character stream and byte stream at the same time?

[Jazlyn Lee]

If the data consists of two type of data: the first line is a character stream and followed by an png image byte stream, how can I use java I/O to read them?

I tried to use DataInputStream from a socket connection and then use BufferedReader to execute readLine(), then use BufferedInputStream to execute read() to save the image, but it doesn't work. I can get the correct characters of the first line while get an invalid image file.

Any suggestions will be appreciated.

Thanks.

[Joe Ess]

You should be able to do it. There's a couple of pitfalls with your approach. DataInputStream is intended for reading Java primitives. If the writer is not a Java program, you could have issues. BufferedReader expects to see the platform-dependent EOL indicator, so if your server is a different operating system, that could be another issue.

[Jazlyn Lee]

Actually the server program which generates the data stream is written in C.

I modified my code as below, but it still cannot get the valid image.

    
        PrintStream ps = new PrintStream(theSocket.getOutputStream());
	ps.print(command + "\r\n");
	BufferedInputStream in = new BufferedInputStream(theSocket.getInputStream());
	BufferedReader r = new BufferedReader(new InputStreamReader(in));
	r.readLine();
	BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream("image.png"));
	int i = 0;
	while ((i = in.read()) != -1) {
		out.write(i);
	}
	out.flush();
	in.close();
	out.close();

Am I on the wrong way?

[Paul Clapham]

Don't use a BufferedReader. If you do that, then the first time you read something from it, it will take 4096 characters (or whatever is the buffer size) from the underlying input stream and store it in a buffer (hence the name of the class). In your case you don't want it to read all those extra characters. So don't use buffering.

[Jazlyn Lee]

Paul Clapham wrote:Don't use a BufferedReader. If you do that, then the first time you read something from it, it will take 4096 characters (or whatever is the buffer size) from the underlying input stream and store it in a buffer (hence the name of the class). In your case you don't want it to read all those extra characters. So don't use buffering.

So what's wrong when I tried to read binary data of the image?

I still can't get the idea.

[Mike Simmons]

Jazlyn Lee wrote:

Paul Clapham wrote:Don't use a BufferedReader. If you do that, then the first time you read something from it, it will take 4096 characters (or whatever is the buffer size) from the underlying input stream and store it in a buffer (hence the name of the class). In your case you don't want it to read all those extra characters. So don't use buffering.

So what's wrong when I tried to read binary data of the image?

I still can't get the idea.

When the BufferedReader reads 4096 chars initially, that probably means it reads 4096 bytes from the BufferedInputStream "in". Let's say the first line takes about 100 characters - then when you call readLine(), the BufferedReader grabs 4096 characters, gives you the first 100 characters as the first line, and saves the remaining 3996 characters in its own internal buffer. Those remaining characters are still available from the BufferedReader, if you make additional calls to read() or readLine() - but they are gone as far as the BufferedInputStream is concerned. So when you go to in.read(), it starts reading from position 4096 (0-indexed). It's already passed on the preceding bytes to the BufferedReader.

(In this discussion I've been assuming a 1-byte encoding so that one byte becomes 1 character. Also I'm ignoring the InputStreamReader, which could also be hiding a few bytes along the way. Those are additional complications. But the basic problem can be understood best from discussing the BufferedReader only.)

What I would do is try to read everything from a BufferedInputStream. Read the first line one byte at a time, writing them to a ByteArrayOutputStream as you go, and looking for a \n. Once you read the \n, you've read the first line, and nothing else. Take the ByteArrayOutputStream, call toArray(), and then make a String from the byte array. That's your first line, as text. Now the BufferedInputStream is ready for reading as an image file.

[Jazlyn Lee]

Mike Simmons wrote:

Jazlyn Lee wrote:

Paul Clapham wrote:Don't use a BufferedReader. If you do that, then the first time you read something from it, it will take 4096 characters (or whatever is the buffer size) from the underlying input stream and store it in a buffer (hence the name of the class). In your case you don't want it to read all those extra characters. So don't use buffering.

So what's wrong when I tried to read binary data of the image?

I still can't get the idea.

When the BufferedReader reads 4096 chars initially, that probably means it reads 4096 bytes from the BufferedInputStream "in". Let's say the first line takes about 100 characters - then when you call readLine(), the BufferedReader grabs 4096 characters, gives you the first 100 characters as the first line, and saves the remaining 3996 characters in its own internal buffer. Those remaining characters are still available from the BufferedReader, if you make additional calls to read() or readLine() - but they are gone as far as the BufferedInputStream is concerned. So when you go to in.read(), it starts reading from position 4096 (0-indexed). It's already passed on the preceding bytes to the BufferedReader.

(In this discussion I've been assuming a 1-byte encoding so that one byte becomes 1 character. Also I'm ignoring the InputStreamReader, which could also be hiding a few bytes along the way. Those are additional complications. But the basic problem can be understood best from discussing the BufferedReader only.)

What I would do is try to read everything from a BufferedInputStream. Read the first line one byte at a time, writing them to a ByteArrayOutputStream as you go, and looking for a \n. Once you read the \n, you've read the first line, and nothing else. Take the ByteArrayOutputStream, call toArray(), and then make a String from the byte array. That's your first line, as text. Now the BufferedInputStream is ready for reading as an image file.

Thanks for your reply.

I have modified my program as below, but it still get an invalid image:

    
public void saveImage(String command) {
  try {
    PrintStream ps = new PrintStream(theSocket.getOutputStream());
    ps.print(command + "\r\n");
    BufferedInputStream in = new BufferedInputStream(theSocket.getInputStream());
    clearEOL(in);
    BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream("image.png"));
    int i = 0;
    while ((i = in.read()) != -1) {
      out.write(i);
    }
    out.flush();
    in.close();
    out.close();
  } catch (IOException e) {
    System.err.println("Err at saveImage(): " + e);
  }
}

private void clearEOL(BufferedInputStream in) {
    byte[] b = new byte[1];
    boolean eol = false;
    try {
      while (in.read(b, 0, 1) != -1) {
        // only stop when we see \r (13) followed by \n (10)
        if(b[0] == 13) {
          eol = true;
          continue;
        }
        if(eol) {
           if(b[0] == 10)
               break;
          eol = false;
         }
       }
    } catch(IOException e) {
       System.err.println("Err at clearEOL(): " + e);
    }
}

Anything's wrong with my code?

[Mike Simmons]

Not offhand. It looks good based on what you've told us. However there are many things you can check to see if they're behaving as you expect. For example: what is the command you're sending to the socket? And what is the response string? It may be an error message telling you there's a problem with the command. If not, look carefully at the last bytes of the text string, and also at the first few bytes of the image file. It's possible there's something there besides the bytes you expect. You can also look up the specification for whatever image format the image file is using, and see if you can interpret the first few bytes according to that specification. Lastly, can you obtain the image file through any other means, so you can compare it to the one your program writes? If you can find exactly how the files are different, that could greatly help in fixing the problem.

[Jazlyn Lee]

Mike Simmons wrote:Not offhand. It looks good based on what you've told us. However there are many things you can check to see if they're behaving as you expect. For example: what is the command you're sending to the socket? And what is the response string? It may be an error message telling you there's a problem with the command. If not, look carefully at the last bytes of the text string, and also at the first few bytes of the image file. It's possible there's something there besides the bytes you expect. You can also look up the specification for whatever image format the image file is using, and see if you can interpret the first few bytes according to that specification. Lastly, can you obtain the image file through any other means, so you can compare it to the one your program writes? If you can find exactly how the files are different, that could greatly help in fixing the problem.

Really thank you for your help.

I made an idiot mistake to use the same Socket InputStream to parse the first line string and then used it again and ignored the first line and saved the image. Therefore, it led the problem of saving the correct data of the image.

The program works correctly now.

Thanks a lot.