• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Reading records into memory or parsing db on demand?

 
James Turner
Ranch Hand
Posts: 194
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi guys,

I have a question about the db file records, do you think it is better to read in the records of the db into an array in memory at the start of the program, or simply create a stream to the file and parse through the file for each find/insert/... request.

Obviously less memory is required to parse the db on demand, but this is also a lot slower and more complex way of doing it. One thing is that the db could grow in the future so reading it into memory could cause problems (assuming this was a real world project).

Does anyone have any views on this issue? With regards to the scope of this assignment, which would be considered best?

Thanx for any help.

Jarvis
 
Hanna Habashy
Ranch Hand
Posts: 532
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi Jarvis:
There are many discussions about this topic in the past few weeks, and there are many intresting opinions.
IMO I prefere to access the data on demand directly from the database. it is slower, but speed is not everything, unless you are doing real time application, like banking. To me, scalability and extesibility is more important. Sure, it takes more programming work, but is is bullet broof solution for extinsible database application. Bringing the data to the memory, IMO is risky. One never know how large the data file is, or how larg the database can grow to. More over, one can never know how large the memory allocated to the JVM at a particular host. All these reasons put so much restirections on how the application can run.
There are many people prefere to bring the database to the memory. You have to use you own judgment, and what you are comfortible with.
 
James Turner
Ranch Hand
Posts: 194
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanx for your view, apreciated.
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic