• Post Reply Bookmark Topic Watch Topic
  • New Topic

reading primitive types from files

 
Ondrej Cervinka
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
DataInputStream and RandomAccessFile have methods for reading primitive types. Unfortunately only bytes can by read into array. The other types must be read one by one.



I use similar code to read terrain elevations from large files. It is quite slow.

Is there any better way to do it?




 
Greg Charles
Sheriff
Posts: 3010
12
Firefox Browser IntelliJ IDE Java Mac Ruby
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The way Java implemented I/O Streams allows you to mix their capabilities together, which can be a really powerful feature. In this case, you need a BufferedInputStream to improve performance. Something like:



That's a stream that reads from the file "myfile.dat" into a 4096 byte buffer, and then parses your data out of the buffer. You get the data interface while still minimizing slow disk I/O calls.

This technique wouldn't really work with a RandomAccessFile though since that lets you jump around the file and read or write.
 
Ondrej Cervinka
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you Greg,

I forgot to mention that after reading the whole file I should end up with array of doubles or DoubleBuffer. I guess your code will be much faster because minimizing I/O access. There is still overhead of copying from buffer (BufferedInputStream) and calling readDouble method really often. The fastest would be reading directly into an array of doubles or DoubleBuffer. This can be achieved by calling once read(byte[]) but only for byte array not double array.
 
Greg Charles
Sheriff
Posts: 3010
12
Firefox Browser IntelliJ IDE Java Mac Ruby
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Interesting. I've never worked with the DoubleBuffer class, but following the API, you should be able to read in all your data into a byte array, then call ByteBuffer.wrap(byteArray). That gives you a ByteBuffer to work with, so you can call: myByteBuffer.asDoubleBuffer(). I don't know how the performance compares with using a buffered DataInputStream though.
 
Ondrej Cervinka
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Interesting too. I will try to compare the performance.
 
Greg Charles
Sheriff
Posts: 3010
12
Firefox Browser IntelliJ IDE Java Mac Ruby
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That's great. Be sure to come back and post what you find out!
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!