• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

concurrent read/writes to db file

 
Greenhorn
Posts: 10
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If a read request for record 4 and a read request for record 5 come in at the same time, both operations should be able to execute concurrently right???
but of course if two threads start moving around the same file pointer, things could get messed up. I've experimented with creating multiple RamdomAccessFile instances (for extra file pointers).
Has anybody thought of this? I am unsure whether this is outside the scope of the project. Just looking for a discussion on the matter.
Thanks
 
Author & Gold Digger
Posts: 7617
6
IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Personally, I create a RandomAccessFile object within a method, so the reference is a local one. No two client threads may modify the same record at the same time (thanks locking), so there is no problem in messing up the DB. I have tested that with 2000 clients trying to book all records of a 500 records DB and nothing got messed up. What I would not do is have an RandomAccessFile instance member and use it to write the file... I prefer to create it everytime the DB file is to be updated...
 
Titus Campbell
Greenhorn
Posts: 10
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So I take it that you made some fairly significant changes to the data class? Do you still call the invariant() function before add(), modifiy()s, and delete()s.
Thanks
 
Ranch Hand
Posts: 2937
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator


but of course if two threads start moving around the same file pointer, things could get messed up. I've experimented with creating multiple RamdomAccessFile instances (for extra file pointers).


Notice that both getRecord() and modify() methods of the Data class are synchronized, so although 2 records can be processed at the same time, only one record at a time will be actually read or modified. That of course assumes that all your clients (threads) use the same object that accesses the same instance of the file, -- and that's how it should be. I think it is a grave mistake to instantiate some other local copies of the file to make it "thread safe".
Eugene.
[ October 02, 2002: Message edited by: Eugene Kononov ]
 
town drunk
( and author)
Posts: 4118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Titus Campbell:
If a read request for record 4 and a read request for record 5 come in at the same time, both operations should be able to execute concurrently right???
but of course if two threads start moving around the same file pointer, things could get messed up.


Titus,
Reads and writes can happen concurrently: After all, only the writing Thread is 'moving' the file pointer. The complexity comes in with writes and writes.
M, author
The Sun Certified Java Developer Exam with J2SE 1.4
 
John Smith
Ranch Hand
Posts: 2937
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator


Reads and writes can happen concurrently: After all, only the writing Thread is 'moving' the file pointer.


I don't think so, -- in the Sun's original code, all the read/write methods of the Data are synchronized. The getRecord() method does move the file pointer by calling the seek() method on the RandomAccessFile object.
Eugene.
[ October 02, 2002: Message edited by: Eugene Kononov ]
 
Titus Campbell
Greenhorn
Posts: 10
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator


Notice that both getRecord() and modify() methods of the Data class are synchronized, so although 2 records can be processed at the same time, only one record at a time will be actually read or modified. That of course assumes that all your clients (threads) use the same object that accesses the same instance of the file, -- and that's how it should be. I think it is a grave mistake to instantiate some other local copies of the file to make it "thread safe".


I agree... Creating multiple RandomAccessFile objects on the same file seems hokey and I probably won't have concurrent reads when I submit the assignment. However, there has got to be an accepted way of reading from two different areas of a flat file at the same time. Is there a way to duplicate the record pointer of a RandomAccessFile without creating a new RandomAccessFile object? I guess I could use FileBuffers from the nio package, but that raises problems other problems.
 
Max Habibi
town drunk
( and author)
Posts: 4118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Eugene Kononov:

I don't think so, -- in the Sun's original code, all the read/write methods of the Data are synchronized. The getRecord() method does move the file pointer by calling the seek() method on the RandomAccessFile object.
Eugene.
[ October 02, 2002: Message edited by: Eugene Kononov ]


Hi Eugene,
Yes and no. even though the methods on the Data class are synchronized, most of the better implementations of the SCJD that I've seen offer each GUI client it's own data class, by using a factory. Notice that the lock/unlock methods on the Data class are not synchronized. This allow you to lock explicitly on another object, say a class-wide, static structure. Point? The synchronization is per client, unless it's a lock/unlock, in which case it's across a single static object.
Basically, calling method a on data object is an atomic operation, per client. This all works out, because you don't need class wide coordination in the case of reads, and you achieve class wide coordination by using lock/unlock in the case of writes (by locking on a static object).
Thus, three different clients could be reading from the db at the same time, even as fourth is updating a record. As a matter of fact, I believe that this is how most operating systems and databases handle this issue(Ever get a message telling you that you can't modify a file because another application is using it? that didn't stop you from reading the file though, did it?). The only real complexity comes in when a record is being modified: which is why we have class wide record locking.
HTH,
M, author
The Sun Certified Java Developer Exam with J2SE 1.4
[ October 02, 2002: Message edited by: Max Habibi ]
 
John Smith
Ranch Hand
Posts: 2937
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator


Yes and no. even though the methods on the Data class are synchronized, most of the better implementations of the SCJD that I've seen offer each GUI client it's own data class, by using a factory.


I am one of those people who offered each GUI client its own remote object, by using a factory, and I got a maximum score on my server design. And yes, my lock/unlock methods are NOT synchronized (there is no need, because they call synchronized methods of the lock manager). However, the key here is that all these remote objects should wrap the same instance of Data, and definitely the same instance of a lock manager. Since the read/write methods of Data are synchronized, there is only one "read" or "write" allowed at a time. Yes, I can see a design where each remote object keeps its own copy of the Data, but this is a sure road to disaster. In fact, my first attempt to model the assignment was that type of design, and my testing revealed that given a sufficient number of simultaneus connections, this leads to unpredictable results in the state of the database. I would recommend this test to everyone:
-- create 20 threads (each thread representing a client), and have each thread repeatedly book 1 seat for the same flight, 100 times for each thread. If the seat count is reduced by exactly 2000 when all threads are done, you pass. I can guarantee that the design where the cient connection keeps its own instance of Data will fail miserably.
But perhaps you are refering to the design where each client has its own instance of Data for reading, but shares the same instance of Data with other classes for writing. That may actually work, but I think the associated complication will hurt the clarity of design.
Eugene.
[ October 02, 2002: Message edited by: Eugene Kononov ]
 
Max Habibi
town drunk
( and author)
Posts: 4118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Eugene Kononov:

I am one of those people who offered each GUI client its own remote object, by using a factory, and I got a maximum score on my server design. And yes, my lock/unlock methods are NOT synchronized (there is no need, because they call synchronized methods of the lock manager). However, the key here is that all these remote objects should wrap the same instance of Data, and definitely the same instance of a lock manager.[/QOUTE]
I think I'm following your design, but it's seems a little round about. Maybe I'm not following you correctly. It seems to me that there is no need for the GUI clients to share a Data object: only that the Data objects themselves to coordinated thought a single shared instance-in your design, that would be the lock manager. I can't begin to speculate on why your extended tests failed without seeing the code: it may be a machine problem, or some other such. But if the locking is implemented properly, there is absolutely no reason (that I can fathom) why instances of the GUI should have to share instances of Data, that I can see.


Since the read/write methods of Data are synchronized, there is only one "read" or "write" allowed at a time. Yes, I can see a design where each remote object keeps its own copy of the Data, but this is a sure road to disaster.


Yes, but the fact that a given Data object is synchronized at the instance level has nothing to do with other Data objects(and, by extension, other GUI clients) being synchronized. It is certainly possible to a given Data object across GUI clients: I suppose that would work. But I don't think it's the better approach.
You only need to coordinate the Data object when you lock(for write activities): and since you are using a lock manager, that should be handled.

When you say 'Data', do you mean a data cache, or a Data object? I wonder if that's where we are missing each other. I mean a Data object: I do not recommend using a cache.


In fact, my first attempt to model the assignment was that type of design, and my testing revealed that given a sufficient number of simultaneous connections, this leads to unpredictable results in the state of the database. I would recommend this test to everyone:
-- create 20 threads (each thread representing a client), and have each thread repeatedly book 1 seat for the same flight, 100 times for each thread. If the seat count is reduced by exactly 2000 when all threads are done, you pass. I can guarantee that the design where the client connection keeps its own instance of Data will fail miserably.


It's a good test. I suggest 100 threads(because they are just as easy to create), and 100 is a nice, round number.
I should say that my own tests did not fail miserably, and that is possible to achieve the highest possible score in every category with such a design.


But perhaps you are referring to the design where each client has its own instance of Data for reading, but shares the same instance of Data with other classes for writing. That may actually work, but I think the associated complication will hurt the clarity of design.
Eugene.
[ October 02, 2002: Message edited by: Eugene Kononov ]



That is not at all what I mean, I mean a simple design, with a single Data object per client: for either reading or writing. In general, the lock method I advocate is very simple(only 13 lines of code, with comments), and the GUI clients are free to use the Data object without any gymnastics. And it has passed the 100 Thread test, several hundreds of times.
I'm curious. Do you know why your original design, with one Data Client per GUI client, failed?
All best,
M, author
The Sun Certified Java Developer Exam with J2SE 1.4
[ October 02, 2002: Message edited by: Max Habibi ]

 
Valentin Crettaz
Author & Gold Digger
Posts: 7617
6
IntelliJ IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So I take it that you made some fairly significant changes to the data class?
I forgot to mention that I was doing the beta exam, and thus, I did not get any Data class but a Data interface which I neither modified nor extended. We don't have any invariant() "function".
 
Max Habibi
town drunk
( and author)
Posts: 4118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That's an interesting approach. For the beta, I created a database driver class, which synchronizes on writes internally. Otherwise, each client gets their own driver. It's not really a different approach then what you have: my driver's simply encapsulates all the messy IO code, so the DataImpl class doesn't have worry about it. Of course, I used scatterers and Gathereres, and not a RAF directly, but it's the same idea.
All best,
M, author
The Sun Certified Java Developer Exam with J2SE 1.4
reply
    Bookmark Topic Watch Topic
  • New Topic