New strategy for avoiding OutOfMemoryException ?
Mike Fourier
Greenhorn
Posts: 25
posted 8 years ago
I've got a program that reads in 400k database rows, and by the end of it, there are 400k new rows in a second table. So it reads a row, does all sorts of lookups/calculations, and inserts one row in a second table.
There was no previous logging code in the method, other than "method begins" and "method ends". Somewhere between those two, I was getting "OutOfMemoryException" (OOME).
After adding a method variable named 'records', within the "while (rs.next())" processing loop, I added this code:
After about 2 minutes, somewhere between records 0 and 10,000 , I got the OutOfMemoryException (OOME). So then I thought to increase the granularity of the logging, and switched the mod to " % 1000". And waiting for about 4 minutes, and on to record 50,000.... no OOME's.
Perhaps I didn't wait long enough. Perhaps if I had waited even more time, the OOME that was *going* to happen, would have happened.
But nevertheless, in 3 trials, switching code back and forth between 1k and 10k, it *always* threw an OOME fairly quickly with 10k, and never threw an OOME at 1k (within the limits of my patience in the test).
So that's a bit strange, right?
There was no previous logging code in the method, other than "method begins" and "method ends". Somewhere between those two, I was getting "OutOfMemoryException" (OOME).
After adding a method variable named 'records', within the "while (rs.next())" processing loop, I added this code:
After about 2 minutes, somewhere between records 0 and 10,000 , I got the OutOfMemoryException (OOME). So then I thought to increase the granularity of the logging, and switched the mod to " % 1000". And waiting for about 4 minutes, and on to record 50,000.... no OOME's.
Perhaps I didn't wait long enough. Perhaps if I had waited even more time, the OOME that was *going* to happen, would have happened.
But nevertheless, in 3 trials, switching code back and forth between 1k and 10k, it *always* threw an OOME fairly quickly with 10k, and never threw an OOME at 1k (within the limits of my patience in the test).
So that's a bit strange, right?
