Win a copy of Murach's Python Programming this week in the Jython/Python forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Alternative to writeShort() in loop  RSS feed

 
Ben Wood
Ranch Hand
Posts: 342
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Probably a reaaaaally dumb question here, please bear with me. I'm generating a 35GB file which is basically a big list of short's. I'm doing this in a loop, and that's a hell of a lot of writeShort() calls, so it takes a very long time.

Is there a way to speed this up I wonder to myself. I'm thinking can I have a buffer like a byte array that I fill up with a bunch of these shorts and then just call write(byte[]) so I'm making less IO calls...but how would I represent each short in the byte array?
 
Joe Ess
Bartender
Posts: 9406
12
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Ben Wood:
Probably a reaaaaally dumb question here


No such thing

The writers of the API anticipated your dilemma. You probably have a series of streams like this:
DataOutputStream -->> FileOutputStream
The cool thing about Java, object-oriented programming, and streams is that we can chain other streams together and change the behavior of the program. For example, if we add a BufferedOutputStream into the mix:
DataOutputStream -->> BufferedOutputStream -->> FileOutputStream
You may make 10000 writeShort() calls, but they get stored up in the buffer of the BufferedOutputStream and only a couple of calls are made to the FileOutputStream.
 
Ben Wood
Ranch Hand
Posts: 342
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Duh! Another triumph for the pea that rattles round my head and doubles up as a brain "Hmm, what I need is some kind of buffer" erm, you meanlike those Buffered streams? Thanks Joe. This has sped up my generation of a 35GB file full of zeroes by a factor of about 20...still, at least the first run before the buffered stream gave my PC something to do over the weekend
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!