OutOfMemory Error

[Alan Shiers]

Hi there,

I've been working on a project that uses Servlets on the server-side and an Applet on the client-side. The applet contains a JList. When loading, the applet sends a request to a servlet for data from a database. The servlet obtains the necessary records and loads a 2D array. I made a serialized class named TableData which wraps the 2D array. The servlet then sends the TableData object out the wire using the writeObject() method. The applet then receives the TableData object with the readObject() method and extracts the 2D array in order to load the JList.

I rudely discovered that this approach only works if the data being sent consists of only 15 - 20 records from the database. Otherwise, anything greater than 20 records, I wind up with this OutOfMemory Error message. This is not good! I hate these kind of gotchas!

I'm sure there must be many out there who have loaded JLists or JTables with data in applets before. Is there a better approach to this problem than the one I just described that avoids this OutOfMemory error?

Alan

[Maulin Vasavada]

Hi Alan,

Here are my thoughts on this,

- To resolve OutOfMemory error you need to analyse your memory requirements in details and figure out any extra memory not being used and try to reduce usage of memory via some simple things like making sure you don't end up creating unnecessary objects and all.

- Try to use -Xmx and -Xms JVM parameters for client JVM (applet jre). Though at the moment I don't recall how to do it for JRE running in the browser ...long time did any applets..

- If your data is reallly huge that comes to applet it won't be sufficient to use -Xmx params to increase the maximum heap size as ultimately you will have machine's RAM restriction anyways..so I would suggest that you should try to see if you can fetch data on demand like Previous , Next functionality or something...

OutOfMemoryError is really program and probably a particular process specific issue. You might see this error at different points in execution at different times..so we have to look for our memory requirements/constraints and take up some apporach or compromise certain things to get things going...

Regards,
Maulin

[Joe Ess]

Originally posted by Maulin Vasavada:

- Try to use -Xmx and -Xms JVM parameters for client JVM (applet jre). Though at the moment I don't recall how to do it for JRE running in the browser ...long time did any applets..

For JRE 1.5, open up the Java Control Panel (On Windows: Start->Settings->Control Panel->Java), click on the tab marked "Java", in the box marked "Java Applet Runtime Settings" click on "View". Enter your parameters in the table cell marked "Java Runtime Parameters". The default maximum heap for an applet is 16 Megabytes.
Keep in mind that changing memory parameters will not cure real memory leaks (i.e. allocating tons of objects and keeping references to the so they cannot be freed). It will only delay their appearance.

[Alan Shiers]

I already tried changing the Java Runtime Parameters property to -Xlx128m but that had no effect. I'm currently trying another approach. Instead of sending all records at once as a 2D array. I'm going to attempt to send the data in the same manner but only one record at a time. This will require a looping structure at each end (server-side and client-side). I've decided on the following code. What do you think?

private String[][] getAllRecords(String requestType)throws NoParameterException, MalformedURLException, IOException, ClassNotFoundException
  {
     if(originalListDataServlet.equals(""))
          throw new NoParameterException("originalListDataServlet");
     String[][] temp = null;
     try
     {    
       URL testServlet = new URL( originalListDataServlet );
       URLConnection servletConnection = testServlet.openConnection();
 
       if (servletConnection instanceof HttpURLConnection)
       {
         ((HttpURLConnection)servletConnection).setRequestMethod("POST");
       }
       else
       {
         System.out.println("this connection is NOT an HttpUrlConnection connection");
         temp = new String[0][0];
         return temp;
       } 
 
       //inform the connection that we will send output and accept input
       servletConnection.setDoInput(true); 
       servletConnection.setDoOutput(true);
 
       //Don't use a cached version of URL connection.
       servletConnection.setUseCaches (false);
       servletConnection.setDefaultUseCaches (false);
 
       //Specify the content type that we will send binary data
       servletConnection.setRequestProperty ("Content-Type", "application/octet-stream");
 
       //send your request to the servlet
       ObjectOutputStream outputToServlet = new ObjectOutputStream(new BufferedOutputStream(servletConnection.getOutputStream()));
      TableData request = new TableData();
      request.setRequestType(requestType);    
      request.setCompanyID(companyID);
      request.setAdminID(administratorID);
      outputToServlet.writeObject(request);        
      outputToServlet.flush();        
      
      //now wait for a response from the servlet
      ObjectInputStream response = new ObjectInputStream(new BufferedInputStream(servletConnection.getInputStream()));
      
      // Read response back from the server
      // A series of TableData objects will flow from the server
      // and each will contain one record from the database.
 
      // This first Object returned will simply indicate
      // the number of records being returned.
      Object count = response.readObject();
      if(count instanceof String)
      {
        int NumberOfRecords = Integer.parseInt((String)count);
        Object record = null;
        TableData td = null;
        String[][] RecordData = null;        
        for(int i = 0; i < NumberOfRecords; i++)
        {
          record = response.readObject();
          if(record instanceof TableData)
          {  
             td = (TableData)record;
             RecordData = td.getData();
             if(i == 0)
             {
               //Dimension temp 
               temp = new String[NumberOfRecords][RecordData[0].length];
             }
             for(int j = 0; j < RecordData[0].length; j++)
             {           
               temp[i][j] = RecordData[0][j];              
             }        
          }          
        }
        record = null;
        td = null;
        RecordData = null;         
        //Lastly, close the streams
        outputToServlet.close();
        response.close();
        outputToServlet = null;
        response = null;
        System.gc();
      }
    }
    catch(NumberFormatException nfe)
    {
      throw nfe;
    }
    catch(MalformedURLException mue)
    {
      throw mue;
    }
    catch(IOException ioe)
    {
      throw ioe;
    }
    catch(ClassNotFoundException cnfe)
    {
      throw cnfe;
    }   
 
    return temp;
  }

[Joe Ess]

Originally posted by Alan Shiers:
I already tried changing the Java Runtime Parameters property to -Xlx128m but that had no effect.

I hope that's a typo.

Originally posted by Alan Shiers:

I'm going to attempt to send the data in the same manner but only one record at a time.

That should make no difference. You are still sending the same amount of data. I'll bet you are doing something in your code to exhaust memory. Try putting some System.out.println's in there. Watch those loop variables. Print out some id to indicate each record being processed. I bet your code doesn't do what you think it does (i.e. in the above code you declare RecordData to be a 2D array, but you only read the first record out of it).

[Alan Shiers]

That should make no difference. You are still sending the same amount of data. I'll bet you are doing something in your code to exhaust memory. Try putting some System.out.println's in there. Watch those loop variables. Print out some id to indicate each record being processed. I bet your code doesn't do what you think it does (i.e. in the above code you declare RecordData to be a 2D array, but you only read the first record out of it).

Yes, RecordData is declared as a 2D array because the getData() method of the TableData object returns a 2D array. That's not a biggy! On the server-side I use the setData(String[][] data) method of the TableData object to load one record. Before I was using it to load all the records. But we all know what happened when I did that.

I can certainly put some println statements inside the looping structure, but all that'll prove is that, if it prints, I got the record. Where she cacks is on the readObject() method. What I'm realizing so far is that when I send TableData over the wire fully loaded with lots of records from the applet to the server using the writeObject() method, that doesn't generate any problems on the way out of the applet. It only seems that when I use readObject() to retrieve a TableData object that is fully loaded that the applet complains about OutOfMemory. This is why I'm thinking of trying things this way, with the looping structure, to receive the TableData object multiple times (containing one record at a time) hopefully without causing an OutOfMemory error. I haven't tried it yet, but I'll let you know how it turns out.

[Alan Shiers]

Well, I'm happy to report that sending a TableData object for every record worked. I no longer get any OutOfMemory errors from the applet. In going through this process, I kept thinking if there is even a better way to do all this. Then the idea of making use of xml came to mind. What do you think? Would it be better to send xml files back and forth with the data?

As it stands now, here's is the code that seems to be working now:

***********Method from the applet***********************

private String[][] getAllRecords(String requestType)throws NoParameterException, MalformedURLException, IOException, ClassNotFoundException
  {
     if(originalListDataServlet.equals(""))
          throw new NoParameterException("originalListDataServlet");
     String[][] temp = null;
     try
     {    
       URL testServlet = new URL( originalListDataServlet );
       URLConnection servletConnection = testServlet.openConnection();
 
       if (servletConnection instanceof HttpURLConnection)
       {
         ((HttpURLConnection)servletConnection).setRequestMethod("POST");
       }
       else
       {
         System.out.println("this connection is NOT an HttpUrlConnection connection");
         temp = new String[0][0];
         return temp;
       } 
 
       //inform the connection that we will send output and accept input
       servletConnection.setDoInput(true); 
       servletConnection.setDoOutput(true);
 
       //Don't use a cached version of URL connection.
       servletConnection.setUseCaches (false);
       servletConnection.setDefaultUseCaches (false);
 
       //Specify the content type that we will send binary data
       servletConnection.setRequestProperty ("Content-Type", "application/octet-stream");
 
       //send your request to the servlet
       ObjectOutputStream outputToServlet = new ObjectOutputStream(new BufferedOutputStream(servletConnection.getOutputStream()));
      TableData request = new TableData();
      request.setRequestType(requestType);    
      request.setCompanyID(companyID);
      request.setAdminID(administratorID);
      outputToServlet.writeObject(request);        
      outputToServlet.flush();        
      
      //now wait for a response from the servlet
      ObjectInputStream response = new ObjectInputStream(new BufferedInputStream(servletConnection.getInputStream()));
      
      // Read response back from the server
      // A series of TableData objects will arrive from the server
      // and each will contain one record from the database.
      //NOTE: THIS APPROACH IS NECESSARY BECAUSE IF I LOAD A SINGLE
      //TABLEDATA OBJECT WITH ALL THE RECORDS AT ONCE, I GET AN OUTOFMEMORY ERROR
      //FROM THE APPLET.
 
      // This first Object returned will simply indicate
      // the number of records being returned.
      Object count = response.readObject();
      if(count instanceof String)
      {
        int NumberOfRecords = Integer.parseInt((String)count);
        Object record = null;
        TableData td = null;
        String[][] RecordData = null;
        //System.out.println("Number of Records: " + NumberOfRecords);        
        for(int i = 0; i < NumberOfRecords; i++)
        {
          record = response.readObject();
          if(record instanceof TableData)
          {  
             td = (TableData)record;
             RecordData = td.getData();
             if(i == 0)
             {
               //Dimension temp 
               temp = new String[NumberOfRecords][RecordData[0].length];
             }
             for(int j = 0; j < RecordData[0].length; j++)
             {           
               temp[i][j] = RecordData[0][j];
               /*
               if(j != 2)
                  System.out.print(RecordData[0][j] + " ");
               else
                  System.out.println(RecordData[0][j]);
               */              
             }                    
          }          
        }
        record = null;
        td = null;
        RecordData = null;         
        //Lastly, close the streams
        outputToServlet.close();
        response.close();
        outputToServlet = null;
        response = null;
        System.gc();
      }
    }
    catch(NumberFormatException nfe)
    {
      throw nfe;
    }
    catch(MalformedURLException mue)
    {
      throw mue;
    }
    catch(IOException ioe)
    {
      throw ioe;
    }
    catch(ClassNotFoundException cnfe)
    {
      throw cnfe;
    }
    return temp;
  }

***********************Server-side Servlet******************

package mvcs;
 
import java.awt.*;
import java.io.*;
import java.util.*;
import javax.servlet.*;
import javax.servlet.http.*;
 
public class RequestTableData extends HttpServlet {
 
    static final String students = "Students";
    static final String instructors = "Instructors";
    static final String courses = "Courses";
 
    public void doGet(HttpServletRequest request,
                      HttpServletResponse response)
        throws IOException, ServletException
    {
      doPost(request,response);
    }
    public void doPost(HttpServletRequest request,
                      HttpServletResponse response)
        throws IOException, ServletException
    {
        ObjectInputStream inputFromApplet = null;
        TableData td = null;        
        ObjectOutputStream out = null;
        String[][] data = null;        
 
        try
        {  
          // get an input stream from the applet
  inputFromApplet = new ObjectInputStream(new BufferedInputStream(request.getInputStream()));
            
          // read the serialized data from applet
          Object o = inputFromApplet.readObject();
          if(o != null && (o instanceof TableData))
          {        
            td = (TableData) o;
            inputFromApplet.close();
            String type = td.getRequestType();
            String companyID = td.getCompanyID();
            String adminID = td.getAdminID();
            /**** OBTAIN THE REQUESTED DATA ****/
            if(type.equals(students))
            {
               data = StudentModel.getAllStudentRecords(companyID);
            }
            else if(type.equals(instructors))
            {
               data = InstructorModel.getAllInstructorRecords(companyID, adminID);
            }
            else if(type.equals(courses))
            {  
               data = CourseModel.getAllCourseRecords(companyID, adminID);
               //System.out.println("RequestTableData - data.length: " + data.length);               
            }             
 
            /**** RETURN THE REQUESTED DATA ****/
            out = new ObjectOutputStream(new BufferedOutputStream(response.getOutputStream()));
            //Tell the applet how many records to expect first
            out.writeObject(String.valueOf(data.length));
            TableData tdata = null;
            String[][] array = null;
            out.flush();
            //System.out.println("Number of Records: " + data.length);
            for(int i = 0; i < data.length; i++)
            {
              array = new String[1][data[0].length];
              for(int j = 0; j < data[0].length; j++)
              {
                array[0][j] = data[i][j];                
              }
              tdata = new TableData(array);
              out.writeObject(tdata);
            }
            out.flush();
            out.close();
          }
        }
        catch(IOException ioe)
        {
          ioe.printStackTrace();
        }
        catch(ClassNotFoundException cnfe)
        {
          cnfe.printStackTrace();
        }
        catch(Exception e)
        {
            e.printStackTrace();
        }            
    }    
}