reading DB records

[Laura Williamson]

OK I can read all the data now.
Is it best to create a list of all records from the flatfile and store this? Or is it best just to leave the flatfile and seek out DB records as you need to access/update them?
The problem I see with keeping a list stored is need to updata data in two places if a chance occurs (in the list and in the flatfile)
Thanks - all advice appreciated.

[Denis Spirin]

Hi Laura

The problem I see with keeping a list stored is need to updata data in two places if a chance occurs (in the list and in the flatfile)

The problem with seeking db file is bad performance.
When you search db, you have to check all records. This happens more often than update; while when you update db all you need is seek() position in file and write new record.
Keeping copy of database leads to better performance and thats what I'd prefer.
[ April 30, 2004: Message edited by: Denis Spirin ]

[Laura Williamson]

OK will go with that
thanks again
LW

[Anonymous]

Laura,
Denis is absolutely right! Two things, though:

performance is not one of the assignments objectives.

if you use RandomAccessFile (which you should, I suggest) you get a little caching for free as the RAF uses an internal buffer i.e. it's very fast. I've tested this with several dozen parallel threads accessing the same RAF instance that reference a file with 3000 records.

I'd recommend you go for the simple solution first and leave the cache aside. You can still add it in the end and wrap it around your existing data class using delegation.
Regards,
Marcel

[Willy Leung]

Hi,
I am also debating whether to cache all records. Here're my thoughts.
I am making the assumption that two specific fields (name + location) in a record make up a primary key. However, the DB interface deals solely with record numbers. So I need to keep a mapping between record numbers and primary keys. If I am cahcing names and locations already, why not the rest of the record and the location of the record in the database? IMO, the cache would make implementing all the methods a lot simpler.
Any comments?
Willy

[Andrew Monkhouse]

Hi Willy,
Welcome to JavaRanch and this forum.
I agree that with the way you are proposing, it will make sense to cache the entire record.
You do need to check that your proposed primary key is logically reasonable given the types of data that can go in them for your assignment (that is, that there is no chance that someone may have a valid reason for wanting to add another record with the same name / location values).
Regards, Andrew

[Willy Leung]

Hi Andrew,
Thanks for the warm welcome and your reply.
After looking at the data in the database file, I do think it is logical to assume that name and location together make up a unique key. I plan to document this in my choices.txt and to go with the caching approach.
Thanks again.
Willy

[Hanna Habashy]

hi guys:
I am also debating between caching the data or using RandomAccessFile stream. My initial thought on caching the record from the database is its size. In real life project, you never know the size of you database, and never know how big the database can grow to. If the database file grow too big, the jvm will run out of memory.
It is the same case dealing with RDB. One can't bring the whole database to the jvm, that is why we have to access the database every time we need to perform a query. I know the project scope is much smaller than real life project, but there are 100 points in general consideration. If the examiner try to test the project with larg size database file, say 150 Mb, and the JVM heap running at 128 mb (defualt), he will get an exception.

[Philippe Maquet]

Hi Hanna,

I am also debating between caching the data or using RandomAccessFile stream. My initial thought on caching the record from the database is its size. In real life project, you never know the size of you database, and never know how big the database can grow to. If the database file grow too big, the jvm will run out of memory.
It is the same case dealing with RDB. One can't bring the whole database to the jvm, that is why we have to access the database every time we need to perform a query. I know the project scope is much smaller than real life project, but there are 100 points in general consideration. If the examiner try to test the project with larg size database file, say 150 Mb, and the JVM heap running at 128 mb (defualt), he will get an exception.

There is a very simple way to handle that issue. It's like a magic, but it works very well: simply wrap your cached records in SoftReferences.
It is guaranteed that the JVM will reclaim any memory used by softly-reachable objects before it may throw an OutOfMemoryError.
Regards,
Phil.

[Hanna Habashy]

hi Philippe:
thanks for the tip. I have never used java.lang.ref package before. I read the api, but I don't have a clear understanding of how to apply this technique to the issue.
If I can store reference to objects, where the object has to live?
for example: if I am reading the db file, and ecapsulate each record in a value object, then referenced this object. The object itself has to be created and stored on the heap, hence using the memory.
Also, consider mutiple users who are accessing different database files. No one needs to reference to the other preson database objects. Each database file will have its corosponding value objects.
clear my understading if I am wrong.

[Philippe Maquet]

Hi Hanna,

thanks for the tip. I have never used java.lang.ref package before. I read the api, but I don't have a clear understanding of how to apply this technique to the issue.

You're lucky enough: among the three types of references, the soft one is the easiest to understand and use.

If I can store reference to objects, where the object has to live?
for example: if I am reading the db file, and ecapsulate each record in a value object, then referenced this object. The object itself has to be created and stored on the heap, hence using the memory.

The object referenced through a SoftReference lives on the heap as any Java object. The trick resides in what the garbage collector *may* do with the object in case there is no other reference to your object than "soft" ones. If it feels short of memory, it may - and *must* before throwing an OutOfMemoryError - clear your soft reference(s) and reclaim the memory used by your object (garbage-collect it).
Practically, lets say that your cache is implemented through a HashMap mapping recNos to records (String[]):
Your Cache class basically needs two simple methods:

synchronized void addRecord(long recNo, String[] record) {
  map.put(new Long(recNo), new SoftReference(record));
}
synchronized String[] getRecord(long recNo) {
  Reference ref = map.get(new Long(recNo));
  if (ref == null) { // record has never been stored in the cache yet
    return null;
  } else {
    return (String[]) ref.get(); // may be null in the case the ref was cleared
  }
}

And any class using your cache will use it this way:

String[] record = cache.getRecord(recNo);
if (record == null) { // never read or cleared by the GC
  record = readRecordFromFile(recNo);
  cache.addRecord(recNo, record);
}
// use record 

Regards,
Phil.
[ May 04, 2004: Message edited by: Philippe Maquet ]