Win a copy of The Little Book of Impediments (e-book only) this week in the Agile and Other Processes forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

nio performance issue

 
Steve Grant
Ranch Hand
Posts: 106
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dear Sir,
I am doing I/O in my project for which I have decided to choose either nio or java.io for file writing and reading . I thought to choose nio because it uses direct memory and gives good performance . I was comparing nio with java.io for writing and reading a file size of 3mb . I was calculating the time for file writing and reading for both java.nio and java.io. But the api of java.nio takes more time for file writing and reading as compared java.io. Following is my analysis:::
For writing file size of 3MB java.nio takes 460 milleseconds and java.io takes 260 milleseconds.

For reading file size of 3MB java.nio takes 375 milleseconds and java.io takes 230 milleseconds.
Anybody can please suggest me better code for java.nio for file reading and writing???
Following is my code :

[Added [code] tags - Jim]
[ January 11, 2004: Message edited by: Jim Yingst ]
 
Steve Grant
Ranch Hand
Posts: 106
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dear Sir,
The above code I am testing on Windows Platform . P3 650Mhz ,512 MB RAM.
Thx ,
Siddharth K
 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There are a lot of different effects here. A few thoughts:
Try reversing the order of your tests. I think you'll find that the part of the problem here is that the JVM is slower as it starts out.
With NIO, it's redundant to have a ByteBuffer and a byte[] array. The time you spend copying from one to the other is the main reason NIO seems to take longer in your tests. With NIO, you usually use a ByteBuffer instead of a byte[] array. Removing the byte[] from your NIO methods makes NIO raster than IO here.
When you read(byte[]) or read(ByteBuffer), you really need to do it in a loop, checking the return value to make sure you've read eveything you planned to. The APIs for these methods make it clear you're not guaranteed to read everything you request all at once. E.g.

or

Incidentally the names of your variables were rather confusing. Why is a FileInputStream called targetFC? Why do you create overloads such as addFile() and addFile(String) rather than giving the methods meaningful names such as addFileUsingIO() and addFileUsingNIO()? It pays to write clearly, even for a test like this.
For fast performance with large files, you may want to try FileChannel's map() method to get a MappedByteBuffer, for reading or writing. Again, you use this instead of the byte[] or plain ByteBuffer you were using otherwise.
Hope that helps...
[ January 11, 2004: Message edited by: Jim Yingst ]
 
Steve Grant
Ranch Hand
Posts: 106
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dear sir,
Sorry for using confusing variable names in my code. Sir the above code will be used in class called VFSIOService which will have addFile and downLoadFile method . Now the addFile method will receive a byte array from an EJB for which I have to create a file and store it on a hard disk. For this I was thinking about whether to use nio or java.io. The same situation is for downLoadFile. For this my java api should read the file from hard disk and return it to tha calling EJB in the form of byte array . So that is why I was dealing with byte array. So is there any effecient way of converting byte array to byte buffer and then using the bytebuffer for rading and writing files.
Thx & Regards,
Siddharth K
 
Spencer Lee
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,

I thought this was the appropriate thread to include my test and test results. I also don't seem be seeing the performance advantages of using nio over simply a BufferedReader to read lines.

To summarize my test, I calc the time it takes to read a 1 GB text file.

Code using the BufferedReader:


Code using nio:


Each execution of my test only ran one of these tests so that the system would not buffer/cache any data of one test for the other test.

The results of my test were as follows:

BufferedReader:
Line read (16777216): 54secs
Line read (16777216): 100secs
Line read (16777216): 64secs

NIO:
NIO direct read lines(16777216): 110secs
NIO direct read lines(16777216): 111secs
NIO direct read lines(16777216): 110secs

I also tried alternative ways to read the file using nio.

  • Replacing allocateDirect with simply allocate
  • Creating a MappedByteBuffer



  • And the results I got were not any more encouraging:

    NIO non-direct: (I did NOT expect the results to be better...)
    NIO non-direct read lines(16777216): 110secs
    NIO non-direct read lines(16777216): 112secs
    NIO non-direct read lines(16777216): 110secs


    NIO Mapped:
    NIO map (16777216): 218secs



    I feel like I am missing a key concept in using nio. Any help/suggestions would be much appreciated.

    Thanks!

    Spencer
     
    Ernest Friedman-Hill
    author and iconoclast
    Marshal
    Pie
    Posts: 24212
    35
    Chrome Eclipse IDE Mac OS X
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Originally posted by Spencer Lee:

    I feel like I am missing a key concept in using nio. Any help/suggestions would be much appreciated.


    I would imagine the reason your java.nio results seem worse than your java.io results is that the technique you're using to read lines with NIO (one character at a time appended to a StringBuffer which starts at zero length for each line) is pretty bad. Have a look at what BufferedReader actually does: it buffers characters into a char[], examines them for newlines, then creates Strings with the whole line's worth of characters at once; it uses StringBuffers only to accumulate data when the character buffer is emptied during a readLine().

    In any case, the best you could hope for is for your readLine() to be about the same as BufferedReader, because low-level file I/O has been rewritten on top of java.nio -- in other words, the performance advantage of nio is already built into java.io.
     
    Spencer Lee
    Greenhorn
    Posts: 13
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    Thank you for your suggestion. I will have a look at the BufferedReader implementation.

    Also, when you say that nio is already built into io... are you saying that there are no real performance reasons to use the nio classes over the io classes?

    Thanks again
    Spencer
     
    Joe Ess
    Bartender
    Posts: 9340
    10
    Linux Mac OS X Windows
    • Mark post as helpful
    • send pies
    • Quote
    • Report post to moderator
    The real reason to use NIO is features, not performance. Have a look at this article that talks about what you can do with NIO that you cant do with standard IO.
     
    • Post Reply
    • Bookmark Topic Watch Topic
    • New Topic