• Post Reply Bookmark Topic Watch Topic
  • New Topic

split file using thread  RSS feed

 
Edward Chen
Ranch Hand
Posts: 798
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I want to split a big file into small chunks using multi-thread. The below is the coding. But I found, more thread will take more time.
In one thread, it will take 13 seconds, but in three threads, it will take 26 seconds.

Any suggestions ?

 
Joe Ess
Bartender
Posts: 9406
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Threads are useful to run operations in parallel which can operate independently (i.e. querying a database and displaying a process bar). They are not useful to try to speed up an operation like performing IO on the same file. The limiting factor on file IO are disk operations (seek, read, throughput). By creating threads to split a file operation up, you are actually creating MORE overhead in the form of more IO requests, plus the overhead of creating threads (allocating stack memory and so on).
What's more, the threads you are creating have the same priority. Threads of the same priority will not interrupt each other, so even if readJob1 was waiting for a read/write, readJob2 could not interrupt it (and we really wouldn't want it to, because they could spend all their time contending for the disk rather than working).
[ August 15, 2007: Message edited by: Joe Ess ]
 
Saurabh Patil
Ranch Hand
Posts: 48
Eclipse IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Joe has explained very clearly. Use threads only when you want to do repetative operations over more than one entity and create that many number of threads as many numbers of entities you have. If more than one thread act on one entity they are simply disturbing each other!

One suggession, you can create one thread for each file accepted from user so that each thread works on division of file inputted by user, so that completion of division operation ends for all files at same time.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!