Files.lines vs BufferedReader vs Files.newBufferedReader java.nio.charset.MalformedInputException

[Markus Schmider]

I have a program which parses files in a directory. The files are supposed to be UTF-8 encoded. However if by mistake a file has another encoding, it is not supposed to crash, but to ignore the file.
I have started out using Files.lines to read the lines. But this does not work as required.
A much simplified version:

Path path = Paths.get("data/fehlerhaft/gebRueck_082350_nonutf8.txt");
try {
Files.lines(path).forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("End");

The file is ANSI encoded. This code crahes, "End" is never printed.

Exception in thread "main" java.io.UncheckedIOException: java.nio.charset.MalformedInputException: Input length = 1
at java.io.BufferedReader$1.hasNext(Unknown Source)
at java.util.Iterator.forEachRemaining(Unknown Source)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source)
at misc.StreamReaderTest.main(StreamReaderTest.java:18)
Caused by: java.nio.charset.MalformedInputException: Input length = 1

Why is the exception not caught?

When I change the code to "pre-Java 8"

FileReader fis;
try {
fis = new FileReader(path.toFile());
BufferedReader br = new BufferedReader(fis);
    String line = null;
    while ((line = br.readLine()) != null) {
    System.out.println(line);
    }
    br.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
   
System.out.println("End");

the program reads and prints all the lines without throwing an exception.

When I obtain the BufferedReader from Files, the program cannot read the file.
The exception is caught. "End" is printed.

FileReader fis;
try {
BufferedReader br = Files.newBufferedReader(path);
    String line = null;
    while ((line = br.readLine()) != null) {
    System.out.println(line);
    }
    br.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
   
System.out.println("End");

Is there an explanation for the different behavior of the BufferedReaders?
Is there any way to make the first version work?

I am using java version "1.8.0_131".

Thanks,

Hans

[Dave Tolls]

 

Markus Schmider wrote:

Exception in thread "main" java.io.UncheckedIOException: java.nio.charset.MalformedInputException: Input length = 1
at java.io.BufferedReader$1.hasNext(Unknown Source)
at java.util.Iterator.forEachRemaining(Unknown Source)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Unknown Source)
at java.util.stream.ReferencePipeline$Head.forEach(Unknown Source)
at misc.StreamReaderTest.main(StreamReaderTest.java:18)
Caused by: java.nio.charset.MalformedInputException: Input length = 1

Why is the exception not caught?

If you look at the first line of that stack trace it is actually throwing an UncheckedIOException, which is wrapping the root-cause MalformedInputException.
You'd have to catch the former to handle this.

[Rob Spoor]

I think the issue lies in the encoding used. The methods in Files use, if none is explicitly defined, UTF-8 encoding. The "old" I/O code uses the system default, which depends on your operating system but is hardly ever UTF-8.

Can you try the following to two bits of code and post the results?

Path path = Paths.get("data/fehlerhaft/gebRueck_082350_nonutf8.txt");
try {
Files.lines(path, Charset.defaultCharset()).forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("End");

FileInputStream fis;
try {
fis = new FileInputStream(path.toFile());
BufferedReader br = new BufferedReader(new InputStreamReader(fis, StandardCharsets.UTF8));
    String line = null;
    while ((line = br.readLine()) != null) {
    System.out.println(line);
    }
    br.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
   
System.out.println("End");

The first uses your existing Files.lines code but uses the system charset instead of UTF-8.
The second uses your existing BufferedReader code but uses UTF-8 instead of the system charset. Note that I had to use FileInputStream + InputStreamReader instead of FileReader for that.