Win a copy of Kotlin in Action this week in the Kotlin forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

How to define compute-intensive and IO-intensive tasks?  RSS feed

 
wd nanji
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I want to use a thread pool to parse multiple XML concurrently,which take String as input.
I've be told that the size of the pool should be tuned differently for compute-intensive and IO-intensive tasks,but how to clearly define whether a task is compute-intensive or IO-intensive? I used a SAXParser to parser the XML and i can't tell exactly which kind it belongs to.
Please help.
 
Adi Pandit
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Some general inputs you might be able to use in your decision making.

I/O intensive tasks are quite straightforward to identify - ones which are reading or writing data to an external resource(disk/db)
So for these tasks the bottleneck is the socket(db connections/file handles and similar resources)
If your I/O task use the same resources your threads should be optimized to the number of these resources available to you and the pooling mechanisms used to access these resources.

Another factor to consider in case of file/db operations is that even though you can get 10 connections but if connection is hitting the same table/file then that table lock will become the bottleneck and spawning 10 threads to use 10 connections will not yield any speed improvements. In some cases it may even degrade performance due to increased concurrency operations.

Again you have mentioned you are using SAXParser so the parsing will be compute intensive but reading the xml and loading into the SAXParser will be I/O. Reading and loading XML will also depend where are you getting the XML from, is it from a file(i/o) is it from a web service (then I am not sure).

In Compute intensive tasks you are not reading or writing but processing data. They are more amenable to parallelization, meaning if you can break them up into independent units of work. Then you can spawn more threads if more cores/cpus are available increase the scalability. Parsing would definitely be compute intensive.
 
wd nanji
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks a lot, I'm clear now
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!