• Post Reply Bookmark Topic Watch Topic
  • New Topic

Performance not improving after using Multithreading  RSS feed

 
Meenal Srivastva
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,
I have an application where i transfer documents from an ECM to another ECM(sort-of!).
For transferring 500 files, without multithreading the time taken is about 20 minutes.
After implementing multithreading (and limiting the max number of concurrent threads to 5 --a system requirement), theoretically the time taken should be around 4 minutes ( 20 divided by 5). But the time taken is still hanging at around 18 minutes.
I am not using any synchronised method or object.

Any suggestions?
 
Ulf Dittmer
Rancher
Posts: 42970
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What does "transfer" mean? Is the process bound by CPU, I/O, or a database?

In general, you can't expect any speedup by multi-threading of processing-intensive single user jobs like the one you describe, at least not on single CPU systems. (Theoretically, it may even be slower, because in addition to the processing, there's also a certain -small- overhead of switching between threads.)
 
Nitesh Kant
Bartender
Posts: 1638
IntelliJ IDE Java MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Meenal:theoretically the time taken should be around 4 minutes ( 20 divided by 5).

It does not really work like that untill and unless the number of processors is equal to the number of threads running on the system.
The reason is that even though the number of threads are more than one, the resource(processor) that execute it is still one(for single cpu boxes) so the threads are time sliced(scheduled) based on their priorities. What it means is that for a single cpu box, at any moment of time, effectively, only 1 thread can be run. All the threads in the system are scheduled by the OS to give you an impression that all the threads are running simultaneously.

Meenal:I am not using any synchronised method or object.

Yeah, you may not be doing that but the libraries you use, jdk code itself *may* be synchronizing.
 
Meenal Srivastva
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I understood the points about synchronising and the reduction factor for execution time not working the way i thought.

By transfer i mean that the files are picked from one remote server, and dumped into another remote server, with some processing done for each file in between.

The threads need to perform I/O in the sense that they have to iterate through an InputStream object to find out its length( available() method does not work for me). So this becomes a processing intensive task in reality.

Also, certain methods which send requests to a remote server are made. These make the threads block/wait.
 
Nitesh Kant
Bartender
Posts: 1638
IntelliJ IDE Java MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Meenal:available() method does not work for me

I think you will appreciate this FAQ entry: AvailableDoesntDoWhatYouThinkItDoes

Meenal:The threads need to perform I/O in the sense that they have to iterate through an InputStream object to find out its length( available() method does not work for me). So this becomes a processing intensive task in reality.
Also, certain methods which send requests to a remote server are made. These make the threads block/wait.


I think you need to profile your application a little to find out how much CPU is it actually using, if for huge load also the CPU is not consumed nearly 100% then that means that your work is not really CPU bound and may get a boost by increasing number of threads. (Oh if you have an upper limit of max 5 threads then probably this is not an option)

BTW, it has become more of a performance question than Threads and Synchronization. I think we can move it to the performance forum.
(Moderators, time to pounce on this one )
[ April 25, 2008: Message edited by: Nitesh Kant ]
 
Meenal Srivastva
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I dont mind that at all!
 
Henry Wong
author
Sheriff
Posts: 22853
119
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
As you probably figured out, threading is not a magic bullet to get performance. Heck, it is not even a magic bullet to even get parallelism. It is amazing how many times I have seen applications that get threaded, and still stay effectively, single threaded. There are tons of other factors that need to be considered.

Anyway, there is no reason to move this to the performance forum -- as there is not enough specific details to discuss. If you like to followup this topic with more specifics, we can continue this discussion at a thread level -- or I can move it to the performance forum then.

Henry
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!