• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Can threading help improve I/O performance

 
Rajagopal Manohar
Ranch Hand
Posts: 183
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,
I need to copy a lot of files form a network location to a local machine. Aroung 500 MB.
Initially I tried to improve the performance using buffering, it worked unfortunately it did not help much when buffering went beyond 10000 bytes.

Then I decided to use threads to import different files parallely to futher improve performance. However it seems to have an opposite effect and it now takes more time.

So does threading really help in case of I/O operations?

ps: can any one suggest other ways to improve I/O performance.

Edited to add: I am using java 1.3
Regards,
Rajagopal
[ June 22, 2005: Message edited by: Rajagopal Manohar ]
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13071
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Threading can help if your problem involves both IO and computation since the computation thread can work while the IO thread is waiting for the operating system to provide the next buffer read. With dual-core or dual-processor systems this can be significant.
Reading and writing binary data will be MUCH faster than trying to work with character streams - so don't use Reader and Writer classes.
Buffers should be some multiple of the operating system file blocks size.
If you can contrive to execute the operating system copy with Runtime.exec it will save the time involved in moving data between operating system buffers and Java program buffers at the expense of setting up the extra process.
Bill
 
Joe Ess
Bartender
Posts: 9312
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by William Brogden:

If you can contrive to execute the operating system copy with Runtime.exec it will save the time involved in moving data between operating system buffers and Java program buffers at the expense of setting up the extra process.


You can also do a direct copy using NIO:
Top Ten New Things You Can Do With NIO: #3: Direct Channel Transfers
 
Rajagopal Manohar
Ranch Hand
Posts: 183
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for replies



but it does not do anything.
when I try some thing like notepad the application opens.
cant we just directly execute a xcopy using this method.

I am working from within an IDE

You can also do a direct copy using NIO:
Top Ten New Things You Can Do With NIO: #3: Direct Channel Transfers


Unfortunately I use java 1.3 and will not be able to use nio .

please help me in making use of xcopy and Runtime.exec()
[ June 22, 2005: Message edited by: Rajagopal Manohar ]
 
Joe Ess
Bartender
Posts: 9312
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Rajagopal Manohar:

cant we just directly execute a xcopy using this method.

No. XCOPY is a command, not a program. Have a look at
When Runtime.exec won't, the section marked " Assuming a command is an executable program "
 
Rajagopal Manohar
Ranch Hand
Posts: 183
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No. XCOPY is a command, not a program. Have a look at
When Runtime.exec won't, the section marked " Assuming a command is an executable program "


Thank you very much.
 
Stefan Wagner
Ranch Hand
Posts: 1923
Linux Postgres Database Scala
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I guess you could improve performance by threads with different physical drives.
Operating on a single drive, the reading head has to jump from file to file which is causing the opposite effect.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic