• Post Reply Bookmark Topic Watch Topic
  • New Topic

Search result with 100.000 objects  RSS feed

 
Brian Grey
Ranch Hand
Posts: 43
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Here is my problem:

In my web application deployed on Tomcat, there is a search page that can retrieve all �objects� from an Oracle database. These �objects� are phone cards objects that have properties like phone number, profile, etc�

The client can have 100.000, 200.000 cards or more in the system. Actually when the client wants to see the result of his search, the system displays it in pages with 15 �objects� (or �cards�) per page. On the top of the page there is a direct link of the other pages. So the client can navigate trough all pages using these links or using the previous/next buttons.
The problem is: when the client doesn�t put any criteria into the search page (meaning that he�s going to get all �objects� (= can be 10.000, 100.000 or more)), the system returns �out of memory� after a few long minutes.
The search functionality returns a collection of objects. A �page iterator� splits it into several pages. I think that the collection that contains the result cannot handle it.
By the way, I�m using Kodo as a �persistence manager�.

Here is the method that retrieves all objects from a search (with an empty filter in this case)
protected Collection execute(Class type, String filter) {

final Extent extent = getPersistenceManager().getExtent(type, true);
query = pm.newQuery(extent, filter);
query.setOrdering(orderBy + " " + (orderAscending ? "ascending" : "decending"));

final Collection result = (Collection) query.execute();
return result;
}

Thanks
 
Greg T Robertson
Ranch Hand
Posts: 91
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The default VM is 64MB if you need more run java -X at the command prompt to see parameters for setting initial and max memory.

Alternatively (and I don't know anything about Kodo) but you could change your page/queries so as to return only the current page results so that a much smaller list is needed to display the current page and requery every time the user selects next/previous/etc
 
Brian Grey
Ranch Hand
Posts: 43
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks for the reply

I'm not sure how I'm going to do the "page/querie" because of the impact. The product is going to be delivered next week.

just another question
Is there a project (or kind of library) in sourceforge that deals with the navigation stuff (icon, link, button... next, prevous, next 10 pages...)
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'd guess it would be slow to open a cursor, read and discard rows up to the requested page. If the user selects page 10, read and discard 10 * pagesize rows, then read & keep the next pagesize rows. Might be worth a try to see just how slow.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!