MalformedInputException in BufferedReader

[Roshini Sridharan]

Hi,

Have a problem when reading a file that was transferred from AS400 through FTP via the following command
------------------
bin
quote type C 943
get source destfile
------------------
This creates a xxx.txt file in the Application Server in Linux.

When the Servlet uses the BufferedReader to read the file it returns the below exception when it tries to read the file..

[10/09/08 17:53:16:621 IST] 00000024 SystemErr     R sun.io.MalformedInputException
[10/09/08 17:53:16:621 IST] 00000024 SystemErr     R at sun.io.ByteToCharUTF8.convert(ByteToCharUTF8.java:262)
[10/09/08 17:53:16:621 IST] 00000024 SystemErr     R at sun.nio.cs.StreamDecoder$ConverterSD.convertInto(StreamDecoder.java:314)
[10/09/08 17:53:16:621 IST] 00000024 SystemErr     R at sun.nio.cs.StreamDecoder$ConverterSD.implRead(StreamDecoder.java:364)
[10/09/08 17:53:16:621 IST] 00000024 SystemErr     R at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:250)
[10/09/08 17:53:16:621 IST] 00000024 SystemErr     R at java.io.InputStreamReader.read(InputStreamReader.java:212)
[10/09/08 17:53:16:621 IST] 00000024 SystemErr     R at java.io.BufferedReader.fill(BufferedReader.java:157)
[10/09/08 17:53:16:636 IST] 00000024 SystemErr     R at java.io.BufferedReader.readLine(BufferedReader.java:320)

This works fine when the file is read from windows OS. Understand that there's some problem understanding the Japanese character while the read. Request your valuable inputs on this .. since we are hang up here.

Thanks in advance for the inputs and time.
[ September 10, 2008: Message edited by: roshini sridhar ]

[Joe Ess]

[10/09/08 17:53:16:621 IST] 00000024 SystemErr     R at sun.io.ByteToCharUTF8.convert(ByteToCharUTF8.java:262)    

This sure sounds like you are trying to read a value that can't be converted into the UTF-8 codepage.
How about setting the BufferedReader to use the same codepage that the file uses, or use a Stream, as they do not do character conversion.

[Roshini Sridharan]

Thanks a lot for the reply.

Can you please tell how to set the codepage for the Reader, also is there any default codepage format for AS/400 files based on the ftp command we used to download to [quote type C 943 ]

Also would like to know how the file same is read when created in the Windows machine.
[ September 10, 2008: Message edited by: roshini sridhar ]

[Joe Ess]

Originally posted by roshini sridhar:
Thanks a lot for the reply.

Can you please tell how to set the codepage for the Reader,

I apologize. I swear that there was some way to change Reader to use something other than the default encoding, but there doesn't seem to be.

Originally posted by roshini sridhar:

Also would like to know how the file same is read when created in the Windows machine.

The only thing I can think of is that Windows is not using UTF-8 as the default encoding.
My advice is use a stream if you are not doing any text processing. It will be faster and you won't run into encoding issues.

[Nitesh Kant]

Joe:
I swear that there was some way to change Reader to use something other than the default encoding, but there doesn't seem to be.

Actually, it is not directly specified in the BufferedReader but in the reader that it is wrapping.
So, if one is using an InputStreamReader(As looks the case in the stacktrace) then the character encoding specified in this reader will be used by the buffered reader. After all, buffered reader just wraps over a reader and does not do the byte to char conversion on its own.
P.S.

[Joe Ess]

Originally posted by Nitesh Kant:
InputStreamReader

Thanks. I must have looked at every other Reader in the API and was questioning my sanity!

[Nitesh Kant]

Joe:
I must have looked at every other Reader in the API and was questioning my sanity!

No problems. It happens!

[anadi kumar]

give second parameter, "UTF-16" or "UTF-8" as encoding type, to the InputStream / OutputStream.


InputStreamReader ir = new InputStreamReader(fileinputstream, "UTF-16");





Regards,
Anadi KUMAR
Architecture and R2ds, Channels and Framework, Software Engineer | HSBC Technology and Services - Global Technology
HSBC Software Development (India)Limited
GLT 4.0 - Panchshil Tech. Park, Block D, S.No. 19, Hissa 2A/11/2, Yerawada, Pune, 411006.
India.

Mobile. [phone]
Email. [e-mail]
Email. [e-mail]

[Gladwin Burboz]

I know this thread is very old but didn't find any satisfying answer and hence adding few comments.


http://download.oracle.com/javase/1.4.2/docs/guide/intl/encoding.doc.html
In Extended Encoding Set look for "Cp943" - "IBM OS/2 Japanese, superset of Cp932 and Shift-JIS"

Make sure your JVM has required charset in "jre\lib\charsets.jar"
Use IBM JRE which will mostly have this extended charset for Cp943

InputStreamReader isr = new InputStreamReader(fileInputStream, "Cp943");

Thanks,

Gladwin
[http://www.linkedin.com/in/gladwinb/]

[anadi kumar]

anadi kumar wrote:give second parameter, "UTF-16" or "UTF-8" as encoding type, to the InputStream / OutputStream.


InputStreamReader ir = new InputStreamReader(fileinputstream, "UTF-16");





Regards,
Anadi KUMAR
Architecture and R2ds, Channels and Framework, Software Engineer | HSBC Technology and Services - Global Technology
HSBC Software Development (India)Limited
GLT 4.0 - Panchshil Tech. Park, Block D, S.No. 19, Hissa 2A/11/2, Yerawada, Pune, 411006.
India.

Mobile. [phone]
Email. [e-mail]
Email. [e-mail]