• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Ron McLeod
  • Jeanne Boyarsky
Sheriffs:
  • Paul Clapham
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

Search result with 100.000 objects

 
Ranch Hand
Posts: 43
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Here is my problem:

In my web application deployed on Tomcat, there is a search page that can retrieve all �objects� from an Oracle database. These �objects� are phone cards objects that have properties like phone number, profile, etc�

The client can have 100.000, 200.000 cards or more in the system. Actually when the client wants to see the result of his search, the system displays it in pages with 15 �objects� (or �cards�) per page. On the top of the page there is a direct link of the other pages. So the client can navigate trough all pages using these links or using the previous/next buttons.
The problem is: when the client doesn�t put any criteria into the search page (meaning that he�s going to get all �objects� (= can be 10.000, 100.000 or more)), the system returns �out of memory� after a few long minutes.
The search functionality returns a collection of objects. A �page iterator� splits it into several pages. I think that the collection that contains the result cannot handle it.
By the way, I�m using Kodo as a �persistence manager�.

Here is the method that retrieves all objects from a search (with an empty filter in this case)
protected Collection execute(Class type, String filter) {

final Extent extent = getPersistenceManager().getExtent(type, true);
query = pm.newQuery(extent, filter);
query.setOrdering(orderBy + " " + (orderAscending ? "ascending" : "decending"));

final Collection result = (Collection) query.execute();
return result;
}

Thanks
 
Ranch Hand
Posts: 91
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The default VM is 64MB if you need more run java -X at the command prompt to see parameters for setting initial and max memory.

Alternatively (and I don't know anything about Kodo) but you could change your page/queries so as to return only the current page results so that a much smaller list is needed to display the current page and requery every time the user selects next/previous/etc
 
Brian Grey
Ranch Hand
Posts: 43
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
thanks for the reply

I'm not sure how I'm going to do the "page/querie" because of the impact. The product is going to be delivered next week.

just another question
Is there a project (or kind of library) in sourceforge that deals with the navigation stuff (icon, link, button... next, prevous, next 10 pages...)
 
(instanceof Sidekick)
Posts: 8791
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'd guess it would be slow to open a cursor, read and discard rows up to the requested page. If the user selects page 10, read and discard 10 * pagesize rows, then read & keep the next pagesize rows. Might be worth a try to see just how slow.
 
And tomorrow is the circus! We can go to the circus! I love the circus! We can take this tiny ad:
Clean our rivers and oceans from home
https://www.kickstarter.com/projects/paulwheaton/willow-feeders
reply
    Bookmark Topic Watch Topic
  • New Topic