• Post Reply Bookmark Topic Watch Topic
  • New Topic

Converting outputstream to inputstream

 
nedry smith
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have an application that I need to write data to an output stream and I have another application to that expects to read this data on an inputstream. Previously we were using a socket, I could write data into a server socket and the client could connect and to a getInputStream on it's socket.

I have a need to remove the socket and provide api access. I have tried using PipedInputStream/PipedOutputStream but the performance is much slower. Some googling seems to reveal that this may be caused by the limited buffer size on these objects as well as the synchronization of threads that is required.

Does anyone have a suggestion for a performance way to go from an output stream to an inputstream. I see Ostermiller's suggestion of using a Circular buffer. Is this going to be faster then using piped streams and can anyone provide a concrete example of it's use.

TIA
Bill
 
Davide Simonetti
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
First of all measuring performance of a multithreaded software that uses streams is a really hard task: it's difficult to find why it's being slow. Can be for network connection or for db connection or because before the software was on two machines.

I don't like the CircularBuffer approach because synchronization isn't handled. I prefer standard Sun solution of pipes. Using two threads can speed up your processing especially in new multi-core processors. If you're sure the bottleneck is the small buffer size of the pipe you can change it simply extending PipeInputStream: it's a protected field.



If you think your problem is thread synchronization you can have a look to the Sun performance tuning guide http://java.sun.com/performance/reference/whitepapers/tuning.html.

After reading your post I made some load test of pipes, using an InputStream that generates random data as a source and a NullOutputStream from Apache commons-io as a sink and that's what i got on my 3y.o. laptop:

bytes: 1M bufferSize:1024 time:126ms
bytes: 1G bufferSize:1024 time:94322ms
bytes: 1M bufferSize:32768 time:92ms
bytes: 1G bufferSize:32768 time:83436ms


That's a 12MByte/Sec throughput. Usually network or db connections are much slower than this.

If you're into this i made a library that ease the use of pipes hiding and optimizing the thread generation (using a Thread pool) and let you set the pipe buffer size. It's very easy to use. Let me know if it fits your needs: http://io-tools.googlecode.com



Best regards.
Davide
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!