Out of Memory error

[Ravindra Nandyala]

Hi All,

I have written a standalone java program that fetches records from the database and calculates the checksum of each table.

The problem I am facing is with tables containing huge data sets, my program resutls in an Out of Memory error.


Below is the error that I get when I run the program with following options :

java -Xms256m -Xmx1536m DatabaseCheck.DBCheckSum

JVMDG303: JVM Requesting Java core file
JVMDG304: Java core file written to /opt/projects/gsd/java/DatabaseCheck/javacore430152.1231233966.txt
JVMDG274: Dump Handler has Processed OutOfMemory.
JVMST109: Insufficient space in Javaheap to satisfy allocation request
Exception in thread "main" java.lang.OutOfMemoryError

I have tried using setFetchSize(Integer.MIN) method to limit the number of records retrieved but this approach also does not work.

Below is sample code of my program:

    select_table_rows = "select * from " + tname ;
		try{
			 stmt = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_READ_ONLY);
			 rs   = stmt.executeQuery(select_table_rows);
			 rs.first();
			 int columnCount = rs.getMetaData().getColumnCount();
			 logMessage("Number of Columns Retireved for Table : " + tname  + "are : " + columnCount  ,LogWriter.INFO);
			 while(rs.next())
			 {
				 for(int col=1; col<= columnCount; col++ )
				 {
					 Object obj = rs.getObject(col);
					 	if(obj == null)
					 	{
					 		continue;
					 	}
					 	if(obj instanceof Blob)
					 	{
					 		    Blob b = (Blob)obj;
					 	        long blobSize = b.length();
					 	        for (int block = 0; block < blobSize/blockSize; block++) {
					 	         bytes = b.getBytes(block*blockSize+1,blockSize);
					 	         digest.update(bytes); }
					 	        bytes = b.getBytes((blobSize/blockSize)*blockSize+1,(int)blobSize%blockSize);
						        digest.update(bytes);

					 	}
					 	if(obj instanceof Clob)
					 	{
					 		Clob c = (Clob)obj;
					        long clobSize = c.length();
					        for (int block = 0; block < clobSize/blockSize; block++) {
					          bytes = c.getSubString(block*blockSize+1,blockSize).getBytes();
					          digest.update(bytes);
					        }
					        bytes = c.getSubString((clobSize/blockSize)*blockSize+1,(int)(clobSize%blockSize)).getBytes();
					        digest.update(bytes);

					 	}
					 	else
					 	{
							bytes=obj.toString().getBytes();
					 		digest.update(bytes);
					 		obj = null;

					 	}
				 	}
				}

If anybody can help me with this then it will be great!

Thanks
Ravi.

[Bauke Scholtz]

You know, each byte of a byte[] allocates exactly one byte of memory?

Can you now imagine what will happen if the byte[] contains more bytes than the JVM available memory?

Either give it more memory or buffer the bytes in an InputStream.