• Post Reply Bookmark Topic Watch Topic
  • New Topic

Producer/Consumer Question  RSS feed

 
Shitij Agarwal
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hey,

I have a question on a slightly modified version of the standard producer/consumer threading behavior.

My producer throws out file a.xml every 0.5 seconds that my consumer needs to process in 1 minute. In a non-multithreaded environment, I would just call the appropriate method in the consumer class telling it to read a.xml and process it. But, what I really want is that in the minute that it takes for my consumer to process my a.xml, I want my producer to throw out another file b.xml and then wait until the consumer finishes processing a.xml. The consumer will then see that there's a b.xml ready and will start processing it. As soon as the consumer starts processing b.xml, the producer will get out of its waiting state and update a.xml and once again wait until the consumer is done with b.xml...and so on.

My biggest hurdle is that I'm not being able to figure out how the producer and consumer files will need to communicate with each other to achieve the above synchronised behavior. Could someone please provide a few starting hints on how to proceed? A few simple lines of code could help a lot too.

Thanks.
 
Sathvathsan Sampath
Ranch Hand
Posts: 96
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I guess you can start off with some asynchrnous callbacks on "file-produced" and "file-consumed" events and eventually see if it makes sense to refactor to observer pattern impelmentation. Let me know if you need more help.
 
Shitij Agarwal
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hey,

I have little idea about what you're saying. I am not even familiar with the observer pattern implementation. What I'm looking for is a simple mechanism just based up wait/notify, where my producer waits after generating an xml file if the consumer is already running, and then when the consumer is done running, then the producer will need to call the consumer method again. Now, will this consumer method need to be synchronized? Basically, what I really don't know is how to have the producer continue to create a new xml file without waiting for 1 minute for the consumer to process the file. Right now, when I call the consumer method from my producer, then I have to wait before the consumer method call returns for my producer to proceed.

Thanks a lot.
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm a little worried that I give the same answer to so many questions, but if you're in JDK 5 look at the thread pooling provided by Executor. For earlier JVMs there are other pools like the Apache Commons and the Doug Lea concurrent packages.

The basic idea with threads is that for every file the producer could start a new consumer thread and then go merrily on its way to produce the next file. A couple concerns come up ... running a zillion threads at once can bog down the JVM, spending more time managing threads than doing any real work. And creating and destroying a zillion threads is also kinda expensive.

That's where pooling becomes attractive. You can create a specified number of threads. The producer puts consumers into a queue, and the threads pull the consumers off the queue and run them. If the consumers run behind the queue will pile up so you need to be sure you can handle the volume. If the consumers get ahead and the queue goes empty the threads just wait around for more work.

Does the producer need to know when the consumer is done? If so you can look at callback mechanisms, something like listeners in Swing.

Does that help?
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!