Win a copy of The Little Book of Impediments (e-book only) this week in the Agile and Other Processes forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

How to realize a POJO model with large result sets from the backend

 
Sebastian Himberger
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Howdy,

this is my first posting in this saloon and i don't know if this is the best location for my question so please give me an advise if i made a mistake. I'm currently planning to develop a small application for personal training (I'm no java guru) and just stumbled over the following problem:

I would like to build my application model with POJO's because it like it having objects that are not tied to specific frameworks or tools. In my application i would have multiple POJOs wich would hold lists. E.g.



These Lists would be filled with data from the backend (e.g. DB). But at the point the Backend contains many entries (maybe 1.000.000 or more) i think it would be inefficient to store them all in memory and perform e.g. sorting operations on them (just for the case the application will have a web frontend, there could be many concurrent requests).

Another problem is that the user mostly needs only a small subset of the complete list (e.g. the 10 newest, 20 per page, only entries starting with A, etc..) but if i rely on the POJO approach i have to store the complete List in memory because the Collections Framework only works with the complete dataset. I thought about using some sort of Controller wich fills the POJOs only with the data they need to display the requested information but would this lead to good design? I want to have the POJOs as loosly coupled to the backend as possible.

How would you encouter this sort of problem? Are there any Java classes/techniques that i'm not aware of or are my requirements just not possible to solve?

Thanks in advance for your ideas!

best Regards

Sebastian
 
Ilja Preuss
author
Sheriff
Posts: 14112
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Holding all your data in memory might not as insane an idea as it might sound: http://www.prevayler.org/

Most people, though, put their data into relational databases, often using a persistence layer. There are quite a few of them working with POJOs, Hibernate probably being one of the most popular from the open source sector.

Moving to Object Relational Mapping...
 
Ilja Preuss
author
Sheriff
Posts: 14112
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Also take a look at http://www.agiledata.org/essays/implementationStrategies.html

And welcome to the Ranch!
 
Sebastian Himberger
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the links and for the welcome.

I will look into it.

Regards
Sebastian
 
David Harkness
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is an issue we all inevitably face. One solution is to model the contained items (the entries in your model) as top-level entities that can be accessed on their own (outside the context of a FooList). This allows you to have methods that return a subset of entries based on your needs.

Also, it often helps to have a concrete example. If there are times where you need the entire list in memory at once to perform a calculation, you can usually find a way around it.

Welcome to the Saloon; pull up a stool.
 
Sebastian Himberger
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi David,

thanks for the welcome *grabs a beer from the bartender*

I know i can get around the problem through making the Entries loadable by themself and probably that's what i'll do in the end. Perhaps i could also offload calculations to the backend and in case it's a database this would make sense.

To provide you with more concrete example is difficult. I currently develop only small modules for existing application (e.g. CMS) and thought about building a small application for training purposes (maybe a forum, or cms - i know that there a lot available but since it's only for myself this should be no problem). Managing large datasets is one of the Problems i will try to solve and since i have only little experience with this i posted this question.

While reading Ilja's links i found out that hibernate promises integration with the java Collections framework and some sort of "lazy" loading. I think i will look into this features.

Regards
Sebastian
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic