handling UTF-8

[Waldle Cai]

Hi all,
I need to transport UTF-8 message transparently as follows:
1. take a UTF-8 document (say a MIME doc)
2. convert it to a stream
3. feed it to some JAVA interface
4. and spit it out to another component
The should be the same UTF-8 input doc.
What we found is that as conversion takes place and message fed to
JAVA, the message is corrupted. Probably due to some characters which are 3-byte long in UTF-8.
Thanks
Waldle

[Jim Yingst]

Is your "Java interface" required to do any processing at all on the data, or is it just supposed to copy the document from one location to the other? If it's the latter, the easiest thing is to avoid Readers and Writers entirely, and just use streams to do your copying:

  public static void copy(InputStream in, OutputStream out)
  {
    byte[] buffer = new byte[4096];
    int count;
    while (count = in.read(buffer)) != -1)
      out.write(buffer, 0, count);
  }

If you need to do any processing on the contents in Java, then you'll probably find it useful to convert the streams to Strings along the way, which is where Readers and Writers come in. If the data is UTF-8 (or any encoding other than your platform's default encoding) you must be careful to specify the encoding with an InputStreamReader or OutputStreamWriter:

    BufferedReader reader = new BufferedReader(new InputStreamReader(inStream, "UTF-8"));