• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

java.io.IOException: Pipe broken

 
Ranch Hand
Posts: 1970
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I wanted to create a PrintWriter that I could give to JavaDB. The idea was that the lines printed to the PrintWriter by JavaDB would get logged to my logger. I thought PipedWriter and PipedReader sounded like good classes to help me with this.

I came up with the code below. It mostly worked, by sometimes suffered from "Pipe broken" IOException, which made it unusable, because the PrintWriter's println() method hung for ages when this happened.

Any ideas why that might have been happening? Any solutions?

This is on Windows XP at the moment, although it has to run on Mac and Linux too. Java is currently 1.4.1, thinking of going to 6 when it comes out properly on Mac.



P.S. There's one or two bits of mildly disguised proprietory code in the above listing, but I hope it's pretty obvious what they do.
[ February 19, 2007: Message edited by: Peter Chase ]
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Most likely this indicates that the reader thread either has completed, or has been unresponsive for an unacceptably long time. One possibility is that the reader thread has not yet started - perhaps all the threads in WorkerThreadPool.LONG_DAEMON_TASKS are busy, and the read task is still scheduled to begin whenever a thread becomes available? Which could take awhile if all the other tasks are long-running daemon tasks.

Another possibility is that the reader thread has completed due to an interrupt(). This could happen because someone has intentionally called stop() - but it could also happen because somewhere in the thread's past life some other component sent it an interrupt, and this may be the first opportunity the thread has had to handle it. Or another component may have merely retained a reference to the thread, and sent the interrupt much later. I would add a boolean flag (e.g. _isStopped) that can be set when stop() is called, and checked when an interrupt is detected. That way you can ignore interrupts sent for any other reason.

I'd also add a little debug-level logging to tell you when the reader runnable starts, and when it stops (for any reason). Does the broken pipe occur before the runnable starts, after it stops, or (unlikely) in between? I would also add a try/catch just inside run() to catch any unchecked throwables and make sure they're logged. Depending on how your logging is set up and what happens to System.err output, it's possible that uncaught exceptions may be overlooked. E.g. if you log all "normal" exceptions with a logger going to a log file, and don't normally have any reason to look at the standard error stuff, you may miss uncaught exceptions which simply bubble to the top of the stack, kill the thread, and print a stack trace to standard error.
 
Peter Chase
Ranch Hand
Posts: 1970
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the suggestions. I will stop using interrupts, and instead use some private protocol, which outside code can't get hold of. I'll also put in more tracing.

The WorkerThreadPool.LONG_DAEMON_TASKS does not have a limited number of threads available, so there should not be a long delay between scheduling and starting the task, although I realise that there are no guarantees with thread scheduling. FYI there are SHORT_*_TASKS pools (not used in this bit of code) that do have a limited number of threads; the reasoning is that if the tasks are short, it's not unreasonable to have to wait for some of them to finish, during busy periods.
 
Peter Chase
Ranch Hand
Posts: 1970
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Unfortunately, none of the above suggestions worked. I still got "Pipe broken" IOExceptions. So I went to look in the JDK sources. Disappointment lurked there.

PipedReader expects to be used by exactly two threads. One thread should do all the reading and another thread should do all the writing. In my application, there is one thread doing all the reading, but writes come from all sorts of different threads. When the first write occurs, PipedReader stashes the Thread object. When that Thread dies, PipedReader says "Pipe broken". Game over.

Sun's own code contains the following revealing comment: -


I'll have to write my own similar classes that don't have this restriction.
[ February 21, 2007: Message edited by: Peter Chase ]
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ah, that's disappointing. Have you considered simply using an existing logging framework such as log4j instead? Might be quicker, and you'd get a lot more functionality. Unless you've got a lot of existing client code that expects to use a Writer. Or the Writer could just append Strings to the end of a LinkedList, while a reader thread pulls them off from the front. With synchronization as necessary of course. Well, I'm sure you can come up with something. Enjoy...
 
Peter Chase
Ranch Hand
Posts: 1970
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I ended up with a simpler scheme. I now have a LogWriter, which is just a Writer, not a PrintWriter. It doesn't need to do fancy threading. Then, to get a PrintWriter, as required by JavaDB, I just construct a PrintWriter with a LogWriter as the wrapped Writer.



Note that it's JavaDB that insists on having a PrintWriter. Would Log4J have given me a PrintWriter for free? I have a lot of code using my logger, so it would be hard to switch everything to another logger. Where using third-party code with a different log system, I write an adapter to mine; I have one for Log4J, JDK Logging, Apache Commons Logging etc.
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ah, I see. I overlooked the JavaDB part of your needs there, so didn't realize why you needed the PrintWriter. No, I don't think log4j has such an adapter built in. Looks like yours will work nicely though. One possible problem I see is that if you use a PrintWriter without autoflush, you quite possibly won't get any flushing at all until the writer is closed. That could use a lot more memory that you'd like. Might be useful to insert a check of the buffer size, and flush whenever it exceeds some maximum. Or just make sure you use autoflush. Anyway, glad you found a simpler solution.
 
Ruth Stout was famous for gardening naked. Just like this tiny ad:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic