• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Jeanne Boyarsky
  • Ron McLeod
Sheriffs:
  • Paul Clapham
  • Liutauras Vilda
  • Devaka Cooray
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

Tests for the Data class/locking mechanism

 
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
just reiterating/expanding on a previous post (Roberto Jan 28, 2009)

first let me thank Roberto for creating the test -- it was nice to have a baseline to work from

there is a problem with the test class which shows up only under certain circumstances and
is dependant on what type of locking impl you chose (i chose map lock and record lock using ReentrantLock).

the problem scenario is as follows (running on a multicore/processor system with # iterations > 5):

Thread A locks record 1 for delete
Thread B locks record 1 for update and blocks
Thread A deletes record 1
Thread A releases lock
Thread B gains lock
Thread B attempts to update record 1 and throws a RecordNotFoundException

.. at this point the test never releases the lock and the test "fails"; yet the locking code is
working correctly.


To fix this, you need to wrap the update call in a try/catch for RNFE and do the unlock in
the catch.

Unfortunately, it's hard to track this particular error in the output logs when running lots
of iterations I ended up rewriting the test in JUnit and redoing the log msgs which
helped a lot.

Hope this helps others avoid the headbanging that I experienced

Cheers,
pedro

(+/- 2 weeks to go before I submit my code )

 
Bartender
Posts: 2292
3
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Howdy, Peter!

Champion, I believe there is something wrong with your lock() method. The thing is, no matter what you are using (ReentrantLock, ReadWriteLock, etc), these tests should complete normally. Please take a look at this thread.

Now, the thing is, in your lock() method, when the Thread gets the lock of the object, you still have to verify if the record is still valid, because it may have been deleted (which is the case you showed). If not, then you should throw a RecordNotFoundException. It should never reach the update() method.
 
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Peter Gough wrote:.. at this point the test never releases the lock and the test "fails"; yet the locking code is working correctly.


Are you sure your locking code is working correctly In your scenario seems to me that when threadB gains the lock on record 1 I would expect a RNFE being thrown, because the record is priorly deleted by threadA.

[edit] beaten by Roberto with 51 seconds

Kind regards,
Roel
 
Peter Gough
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Absolutely sure my locking code works correctly

As stated previously, the "bug" only shows up depending on the way you implement locking.
I chose a real simple locking scheme that requires the caller to follow proper sequencing:
lock/operation/unlock. Hence, when Thread B gains the lock, the code subsequent to the
blocked recordLock.lock() call does NOT do any recNo checking. Certainly I could add code
that verifies that the record exists after the recordLock is successful, but that duplicates
the check in the update() method (niggling point, i know).

"Sample" code follows :



Regards,
pedro
 
Roberto Perillo
Bartender
Posts: 2292
3
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Howdy, Pedro!

Certainly I could add code that verifies that the record exists after the recordLock is successful, but that duplicates the check in the update() method (niggling point, i know).



Champion, that verification has to be done in your lock() method. If a record is not valid anymore, it can't be locked, and a RecordNotFoundException must be thrown.

Again, it doesn't matter what you are using, these tests should complete normally. The thing is, we have an API, and we use it. It doesn't matter what is underneath it. All we have to do is know the API, use it properly, and the component must handle these situations properly.
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Pedro,

I agree completely with Roberto's opinion. And I would take it 1 step further: update/delete/unlock should NEVER throw a RNFE, because you have to lock a record and after successfully locking it, another thread can't do anything with it (so certainly not deleting), otherwise there would be something wrong with your locking mechanism. That's why I created an own interface (extending Sun's interface of course) and I redefined these 3 methods without the throws RNFE clause (and of course I added an explanation in my choices.txt about it).

Kind regards,
Roel
 
Peter Gough
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you both for your input! I appreciate your candor

I would suggest that the comments in the SUN-supplied API can be interpreted in different
ways (on purpose, of course). Hence, it is hoped that the user of the Data class calls lock()
first, but may skip it and call update() directly (which is why I chose to throw RNFE in my
version of update()). That said, as I'm the one calling the Data class routines, lock() will always
be called first

I think the RNFE on a lock() once the 1st lock() is released comes down to temporal physics.
... and we all know what happens when we bring the time-space continuum into the picture ;)
The record DID exist when the lock was attempted, it just didn't exist AFTER the lock was granted.
But split hairs aside, I agree with your recommendations and will duly add the 2 lines of code to my
lock() method.

Thanks again for the excellent input.

Have an e-pint on me
pedro

 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Peter Gough wrote:Hence, it is hoped that the user of the Data class calls lock() first, but may skip it and call update() directly (which is why I chose to throw RNFE in my version of update()).


You can see your Data class as a kind of API (that's what I did) where other developers have to work with and/or extend its functionality. So you have to prevent abuse of this API (and force correct usage). If someone tries to call an update without owning the lock on that record, he abuses the api (because lock-update/delete-unlock is the way to go) an IllegalStateException is thrown from my update-method. That way no one can update a record without owning its lock.

If you throw a RNFE when recNo is invalid in update-method. What happens if I call your update-method with an existing recNo but without owning the lock on that record?

Kind regards,
Roel
 
Peter Gough
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You are putting behavior into the Data class that was not specified in the interface comments -- which is perfectly fine

My choice was to implement the interface directly as stated and not require locking for find/update/delete. This puts
the onus on the next layer up to enforce transactional behavior (I commented this design, of course ). Hence,
the user is free to choose how to use the class. Let's say the customer wants to upgrade the nightly report program
mentioned in the reqs. That program may have a single thread and may not need to use locking; yours forces them
to use locking while mine doesn't. I'm not saying my approach is better -- just different yet valid.

As for the other issue, here's a contrived example that will hopefully describe why I initially didn't recheck the
record's existence after gaining the lock after blocking:

Out on the JavaRanch, 2 ranchhands, Roel and Roberto , wanted a tall glass of fresh milk. Conveniently, the ranch has a
barn where you can milk cows. Each cow has a number and is in their own locked milking room. The gents know that cow #1 has
the best milk and they head over to the waiting room. They ask the attendant if the cow has milk and she says yes. The single
door key is on the table. Roberto edges Roel out and gets the key, opens the door and heads in [lock(1) by Roberto]. Roel sits
down and waits [Roel is blocked]. Roberto comes out with a glass of milk [delete/update] and puts the key back on the table [unlock(1)].
Roel gets the key [unblock].

Let's say that Roberto took the last of the milk (but doesn't tell anyone). Now Roel has a couple of options: he could go in and try to
milk the cow, find out there's none and leave [RNFE in delete/update] or he could ask the attendant if the cow has milk, find out
there's none and leave [RNFE in lock(1)].

Initially, I chose the former methodology as it felt more "queue-based" or "ticket-based"; but you and Roberto convinced me the latter
is more pragmatic.


Regards and thanks for the thought-provoking discourse,
pedro
 
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello everybody,

I have a concern about the way that I implemented the Data class.

The thing is, I decided to not use cache and every operation manipulate the file. This way, my file became a mess when the threads starts to access the file concorrently.

What should I do? Do I have to change to "synchronized" all my methods that manipulates the file (almost all the method in this class that uses the RandomAccessFile.seek)?

Thanks!
 
Roberto Perillo
Bartender
Posts: 2292
3
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Fala Rodrigo!

Champ, you'll have to synchronize somehow somewhere. One option is to synchronize all methods of the Data class (which is what I did, and my good buddy Roel did the same as well). Another option is to synchronize the DataInput object that you use in your Data class when you read the physical .db file.

Please run some tests and let us know how it went!
 
Rodrigo R. Branas
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Fala Roberto,

I had to fix some points in my code and after that I ran 1000 and it takes just a couple of seconds to terminate!

I synchronized public methods in the Data class and also try a database read before lock the record.

Now everything is perfect!

Thanks for sharing this test scenario!
 
Ranch Hand
Posts: 221
Scala Python Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Roberto and Everybody else,


I've been doing some testing using Roberto's DataClassTest.java and sometimes randomly I run into a deadlock in Eclipse or NetBeans
just by doing 1 iteration.

I saw the comments by Payal Shah in this thread and I wanted to hear your thoughts about calling my unlock() method in the finally block
since my unlock() method can throw RecordNotFoundException?

In my view adding another try block inside the finally block looks kind of cluttered and ugly.


Just in case the locking methods in my interface:




Thanks in advance,


Carlos.
 
Roberto Perillo
Bartender
Posts: 2292
3
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Howdy, Carlos!

I've been doing some testing using Roberto's DataClassTest.java and sometimes randomly I run into a deadlock in Eclipse or NetBeans
just by doing 1 iteration.



I guess it is when you try to unlock a record that was already deleted? Well, if so, I'd say that maybe the way you implemented your unlock() method may be wrong. Remember that, after calling the delete() method, the record will not exist anymore, so you have to be careful there. In which case you throw a RecordNotFoundException in the unlock() method?
 
Carlos Morillo
Ranch Hand
Posts: 221
Scala Python Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Roberto,


Precisely after I clicked Submit in my previous posting I thought the same and rearranged my unlock() method code and now I am able to run your DataClassTest with 100 iterations successfully!



I still had to modify your DataClassTest putting a nested try and catch block inside the finally block where I call my unlock method.


Thanks and Best Regards,


Carlos.
 
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Arg. My new 2010 assignment is 1.1.1 which has no "Data" class. I wonder if they are reverting back to 1.1.1 for all new applicants for now.

I'll see how much effort it takes to back-port this code. I hope the Sun-supplied interface isn't so different as to make back-porting impractical (in comparison to writing new tests from scrath).
 
Roberto Perillo
Bartender
Posts: 2292
3
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hey, champion.

So what's your assignment? URLyBird or B&S?

You don't have a Data class? Is there an interface you have to implement? I guess it has methods like lock(), unlock() and update(), right? Well, if there is in an interface to be implemented with these methods, and this interface has to be implemented with a name other than "Data", then how things work are still pretty much the same as always, just the name of the implementation is different.
 
Blaine Simpson
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Roberto Perillo wrote:Hey, champion.

So what's your assignment? URLyBird or B&S?


URLyBird


You don't have a Data class? Is there an interface you have to implement? I guess it has methods like lock(), unlock() and update(), right? Well, if there is in an interface to be implemented with these methods, and this interface has to be implemented with a name other than "Data", then how things work are still pretty much the same as always, just the name of the implementation is different.



Oops, I missed the requirement that my DB impl. must be named "Data". Sorry. I haven't made it through much of your code yet because I'm back-porting TestJarSubmission.jar first. I'm quite handy with Ant and just want external validation that I haven't missed any bundling requirements. UPDATE: TestJarSubmission working and has earned its pay; DataClassTest building and running with cookie==rowId hack described below.

I must implement the supplied interface sunceritfy.db.DB.

Looking good. It looks like my DB interface is compatible enough with your DBMain interface. It has the methods you listed and based on discussions here it looks like the required params and throw behavior are at least close.

Before looking at the Internet at all, I have the all I/O operations working with my text console driver, purposefully delaying lock enforcement until I have some tests to satsify.

I read in one of these topics that you keep all records in RAM, but elsewhere that you use the RAF pointer for individual read()s or something similar. Maybe reading all into RAM is just for your tests or for your quick-start db-file-loader class, since while keeping all records in RAM would probably be more than adequate for a real app that uses a flat file with tiny records, Sun would probably prefer to see that we know how to write scalable code too. Do you cache at the String[] level or the Room-record level, and do you cache all non-deleteds, lockeds, nothing, or something else? I think I read that you lock on your lock map entries. If so, could you opine on exactly what to lock on. I.e. the Map.Entry, the Long, the String[]/Room-record? I am wondering if locking on a Long would have side-effect issues since Longs are immutable and a future developer could lock Longs for an entirely different purpose. I think locking the Room-records would be at too high a level to satisfy the Data/DB interface. (I'm ignoring the relatively trivial aspect of locking the RAF).

I need to re-read the posts above and related topics, so please just ignore questions which have already been answered or covered.
 
Blaine Simpson
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Significant difference.

It looks like my supplied interface requires to pass a lock id/cookie for methods which require an existing lock for their operation, whereas yours does not. I thought those params showed poor encapsulation when I first saw the supplied interface. Why require the client-side of the API to keeping track of lock "cookies" (as they are called in some of the method JavaDocs), when only one lock can ever be active at any time for a single record.

So, if I were making a real app, I'd ignore the lock cookie params and return values altogether (and remove them from the API if portability concerns allow for that), but here I think it likely that Sun expects us to honor the spirit of the API and use the required params and return values.

My current code (written before I even visited this site) just increments a static counter singleton and requires lock usage to specify the id that was allocated when that lock was obtained. On the one hand, the only benefit to this is as a poor substitute for real client validation; on the other, I wonder if even this will satisfy Sun's intented usage of the cookies.

Any suggestions?

Until I figure out how to satisfy Sun on this, I'll change my lock id code to just use the record id. Otherwise I'd have to add more new tracking code to your DataClassTest than I want to (considering that I will probably change my cookie design before I'm done).
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Blaine,

You have 2 assignments: URLyBird (about rooms) and "Bodgitt & Scraper" (about contractors). Each of these assignments has a number of versions. All these versions have some small differences, but the main idea (and your task to pass scjd) is in every version the same.

You have for example at least 2 interfaces: DB or DBMain. Methods in these interfaces can be create, delete, read, find. But in another interface there is createRecord, deleteRecord, findByCriteria.
Sometimes the signature is different:
So it's completely normal that your situation differs from the one used in the test cases (and other code you'll find on this site). But with the help of your common sense you 'll certainly be able to make the necessary changes.

My instructions say

Your data access class must be called "Data.java", must be in a package called "suncertify.db", and must implement the following interface

And I'm quiet sure your instructions have similar "must" requirements.

A whole lot of your questions have nothing to do with this test case (and possible test results or problems when running thise test). Please adhere to the "how to ask questions on JavaRanch" policy, which can be found here. If you have questions about using a record cache (keeping records in RAM) or how to make your data access class thread-safe,...: 1st use the search engine and try to find an answer on your question. If the answers you'll find don't satisfy you (or you can't find any answer), don't be afraid to start a new thread and ask your question, describe your problem. The JavaRanch community will be glad to help you.
And also don't forget to have a look (and a read) of the ScjdFaq, also very helpful.

Kind regards,
Roel
 
Blaine Simpson
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Roel De Nijs wrote:Hi Blaine,

...
You have for example at least 2 interfaces: DB or DBMain. ...


If you are using "interface" in the general sense, then yes. If you mean it in the specific sense normally used when one is trying to determine exaclty what interfaces and classes are required, then that is incorrect. There is no mention in my assignment for any interface other than DB. I am using another interface (and will add more), but that is not to satisfy any"must" requirement.

I am simply trying to figure out which classes and interfaces in these test classes correspond to my items. And also how to add client-side cookie-tracking to DataClassTest (a significant use case which is required for the 1.1.1 interface but seems to be entirely unnecessary for 1.3.1).

If Sun is indeed reverting URLyBird assigments to 1.1.1 for all new assignments, many other people will be in the same situation as me: they will need to modify Roberto's classes significantly. I see from posts here that 1.3.1 URLyBird applicants can use the supplied classes just by changing a couple lines. I thought there would be general interest in lowering the much higher investement required for 1.1.1 users. Most of my questions about class names and signatures are about getting Roberto's classes working for 1.1.1, and I believe this is the appropriate forum for that.
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

You have for example at least 2 interfaces

Meaning all possible "must implement" interfaces in the different versions of both assignments. Your instructions will mention 1 of these (in your case interface DB, in my assignment it was DBMain).

I had the same assignment as Roberto: the Data mentioned in his test case is simply the data access class which must implement the interface given by Sun (in my assignment DBMain).

I believe Sun is just providing the assignments randomly: you get UB 1.1.1, another developer will get B&S 1.2.1,... In the ScjdFaq you could have read this entry.

Kind regards,
Roel
 
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

And also how to add client-side cookie-tracking to DataClassTest (a significant use case which is required for the 1.1.1 interface but seems to be entirely unnecessary for 1.3.1).



The client side cookie tracking is necessary for both actually, it is just that given the interface for different versions, developers will have to deal with this situation differently, depending on the version of the assignment they received. I got the version 1.1.1 myself as well so you should have these methods defined:



I believe for version 1.3.1, the method declarations are as follows (someone else can confirm):



What this means is that whoever received the version 1.3.1 had to build a structure which supports maintaining client information, whereas the developers which received version 1.1.1 have it somewhat easier, since the cookie value is part of the method signatures. Either way, you need a way of knowing who the clients are. With respect to Roberto's test class, instead of writing these lines:



You would write something similar to:



they will need to modify Roberto's classes significantly.



I didn't have to update Roberto's class significantly. I just changed maybe 10 lines, and really it wasn't so much as changing as it was retrofitting certain method calls to match my interface, but it still took less than 1 minute to change. Think of this class as a helper test. You should still try to build some of your own unit tests, but Roberto's class is perfectly valid for all developers working on, I believe, any version of the SCJD assignment. He even mentions somewhere that you may have to modify where exactly unlock is called from, for example in the try{} or finally{}, but his test case is very good and perfectly valid and I certainly didn't have issues modifying it.

Cheers,

Vlad
 
Roel De Nijs
Sheriff
Posts: 11604
178
Hibernate jQuery Eclipse IDE Spring MySQL Database AngularJS Tomcat Server Chrome Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Vlad Djordan wrote:I believe for version 1.3.1, the method declarations are as follows (someone else can confirm)

The exact signature of these methods in URLyBird 1.3.1 are:
Kind regards,
Roel
 
Greenhorn
Posts: 24
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Roberto,

The locking test class is really a big help and saved a lot of time.

Is there any other test available that someone wrote particularly for Bodgit & Scarper ?

Take care.

Best Regards,
Shahriar
 
Roberto Perillo
Bartender
Posts: 2292
3
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Howdy, Shahriar!

I'm very glad it is being helpful to you!

Is there any other test available that someone wrote particularly for Bodgit & Scarper ?



Hum... I don't know. But, since the interface is pretty much the same (with small variations between a few versions of it), these tests should require only minimal edition to work with any version!
 
Ranch Hand
Posts: 59
Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Roberto Perillo wrote:...these tests should require only minimal edition to work with any version!


Yes, they do!!!
Hey Shahriar! I got that B&S reqiurement and it works with only a few changes.

Regards
Bernd
 
Blaine Simpson
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This test class became much more useful to me after I made some enhancements.

The class runs very verbosely, for understandable reasons, and makes it very easy to miss critical problem reports. Some method invocations are expected to fail at their assigned task (due to trying to lock random record numbers when that record number may not exist at that point in time, etc), and there is nothing to keep track of whether these methods ever succeed. More often than not, I want my test to run without interaction: I want to invoke it from Ant or IDE and have it finish itself, clearly notifying me if there is a problem. I understand that most users of this class haven't had a need for these features, and that's great. For those who would find the test class more useful with these things taken care of, the following enhancement tactics worked great for me.

  • After starting up all the bg threads, poll the Thread.activeCount() every .2 s. or so and quit when == 1; or when the max time limit is reached (20 s. works for me), print a clear message that a dead-lock has occurred and produce a process thread report dump.
  • Add counter variables for methods which may fail in successful situations, so that upon exit you can report how many times these methods have succeed and failed.
  • Switch the printlns to JUL logging statements. Use multiple purpose-specific java.util.Logger loggers so you can easily turn on and off different types of reporting. Otherwise it is takes a lot of tedious study to see details about a particular action type (I'm not talking about logging severity/level, which I know can be filtered easily with a single Logger).
  • When the loop (from the first item above) finishes, display a summary of open locks using a method which you will hopefully have implemented in your Data class, along with the counter values discussed above. If you start with database records in a particular state (no time to elaborate), 0 counters will indicate a serious problem which would otherwise remain unreported whereas all successes indicate your methods are succeeding where they should sometimes fail. The final open lock report itself alerts you to problems of locks not being released in certain contention situations which would not be detected or reported by the test class otherwise.


  • I won't upload my modified test class code, as I made significant changes to the exception catching (and lack thereof), and many of these changes were to accommodate features not required by the project specs (most significantly, my Data.delete() method automatically unlocks after a successful deletion, since no good and much evil can result if that is left to the caller).
     
    Roberto Perillo
    Bartender
    Posts: 2292
    3
    Eclipse IDE Spring Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Blaine,

    the main idea of this class is to be as simple as possible, without requiring ANT or anything else, for instance. You get it and run from the command line. If you run the loop a couple of times, and your program doesn't complete, then you have a deadlock. The developers can use this class, and are of course free to do some customization, but it shouldn't require much changes to become useful with any of the available versions of the interface provided by Sun.

    If you feel that you have something to add in terms of code, you can post it here. Anything that can be helpful to everybody is welcome here.

    Blaine Simpson wrote:(most significantly, my Data.delete() method automatically unlocks after a successful deletion, since no good and much evil can result if that is left to the caller).



    So your delete() method deletes the record, unlocks it and notifies all waiting threads? Well, I'd say that it is better to leave this logic only in the unlock() method. The delete() method should be responsible only for deleting records. It is expected to call unlock() after calling update() or delete().
     
    Blaine Simpson
    Greenhorn
    Posts: 6
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    Roberto Perillo wrote:Blaine,

    the main idea of this class is to be as simple as possible, without requiring ANT or anything else, for instance.



    Agreed, about keeping it "as simple as possible". Like I said, most users have no need for the enhancements. I've listed the additional cases that I personally wanted and I think people can decide whether or not they also would benefit from those use cases. As far as "without requiring ANT or anyting else," I have done absolutely nothing to require Ant or anything other than a JDK, just like your original test class. My class is suitable for letting it run without user intervention and is therefore more suitable for invocation by Ant or IDE, but still enjoys the stated benefits if run interactively.

    Possible exceptios where the average user may benefit are the described situations where your class does not detect failures or false positives in those method(s) which are sometimes expected to fail (depending on real-time app data state); and the failure to notice locks left open after all threads have completed their work.

    Roberto Perillo wrote:

    Blaine Simpson wrote:(most significantly, my Data.delete() method automatically unlocks after a successful deletion, since no good and much evil can result if that is left to the caller).



    So your delete() method deletes the record, unlocks it and notifies all waiting threads? Well, I'd say that it is better to leave this logic only in the unlock() method. The delete() method should be responsible only for deleting records. It is expected to call unlock() after calling update() or delete().



    My delete() method has no redundant code, but it calls unlock() internally if and after it successfully deletes. I completely agree that it makes sense to not do this for the update() method, because the caller may or may not want to do more work with the lock (and that is why I do not unlock in that method). For the case of deleting, allowing the user to delete and leave the lock open just adds unnecessary lock contention (instead of closing it as soon as it is impossible for it to provide any further benefit) and opportunity for Data user mistakes with absolutely zero benefit. Doesn't bother me if you disagree. I described my delete implementation only to justify why I am not posting my code here (i.e. because it has been changed to accommodate this non-required behavior which I decided to implement).
     
    Greenhorn
    Posts: 10
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hey Everyone,

    My sincerest gratitude to this forum in guiding me through my SCJD assignment, it probably would have taken me even longer without it! And thanks to Roberto for posting this multi-threading JUnit test!

    I'd like to contribute a small modification I made to give me an additional "warm & fuzzy" that my code will not deadlock. I added a countdown latch that the main thread in the startTests() method waits on before printing the message "SUCCESS! No deadlocks occurred."


    My startTests() method now looks like:

    Then, at the end of each Thread's Run() method, I added

    The main thread will now wait until all 50 threads have counted the latch down to 0, which can only happen when there is no deadlock.
     
    Roberto Perillo
    Bartender
    Posts: 2292
    3
    Eclipse IDE Spring Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Howdy, Oliver!

    Champion, that's what I'm talking about! There are some cool stuff that can be added to these tests, and I'll do it when I have the time! But everybody will still be free to make suggestions!

    Thanks for the contribution! It will certainly be helpful to everybody!
     
    Ranch Hand
    Posts: 92
    Eclipse IDE Spring Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi folks,

    I slightly rejigged Roberto's excellent DataClassTest for Bodgitt and Scarper. The code below is what I came up with and while it slightly leans towards my implementation (my Data is a facade and the underlying Database relies on a Schema) I hope someone else will find it useful;

    Big thanks to Roberto for providing the original and allowing re-use and modification. Likewise please re-use / modify this for your own testing purposes.



    Any thoughts, problems just let me know.

    Regards,

    John
     
    Roberto Perillo
    Bartender
    Posts: 2292
    3
    Eclipse IDE Spring Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator

    John McParland wrote:I hope someone else will find it useful



    It will certainly be helpful to a lot of people!

    Thanks for the contribution, champion!
     
    Ranch Hand
    Posts: 383
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Roberto
    ,
    I am using the test class you wrote, thank you for it, it already helped me find a bug (locking unrelated, but still). I had to change it a little to suit my needs (I have a cookie, I also moved unlock() to the finally block as update() might throw exceptions) but it is a great place to start and build upon. Once again, thank you for sharing.

    I have a question though:

    /*
    * An exception cannot occur here, otherwise, the unlock
    * instruction will not be reached, and the record will be
    * locked forever. In this case, I created a class called
    * RoomRetriever, which transforms from Room to String array,
    * and vice-versa, but it could also be done this way:
    *
    * data.update(recNo, new String[] {"Palace", "Smallville", "2",
    * "Y", "$150.00", "2005/07/27", null});
    */

    I might be missing something, but this comment seems to be a mixture of two unrelated thing. How come a RoomRetriever can help with the deadlock?

    Regards,
    Raf
     
    Roberto Perillo
    Bartender
    Posts: 2292
    3
    Eclipse IDE Spring Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Howdy, Raf!

    You are very welcome, champion. The idea is to give people a starting point, and each candidate can make the changes according to their needs!

    If the data.update() instruction is reached, then it is guaranteed that an exception will not occur (as it could only occur when calling the lock() method). But the thing is that I should have put the "In this case..." phrase in a new line, I guess that's what caused the confusion. But it is about two different things. My RoomRetriever class has a method that returns a String array, given a Room object. But it could be done by creating the String array "dynamically".
     
    Greenhorn
    Posts: 4
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    In case running the test leads to deadlock, I recommend using the jstack utility that comes with jdk 5+.
     
    Ranch Hand
    Posts: 57
    Oracle Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Hi Roberto,

    Just tested my Data class with your test case and it worked fine. I tried it with up to a 10 million iterations and it worked fine- terminated successfully. Executed it in Eclipse and it terminated successfully. It didn't take too long to run at all, the 10,000,000 iterations took less than a minute to run. I simulated a deadlock by commenting the unlock method in the updating thread class.

    Thanks for such a useful tool



    Cheers
    Olu
     
    Roberto Perillo
    Bartender
    Posts: 2292
    3
    Eclipse IDE Spring Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Howdy, Olu!

    Just tested my Data class with your test case and it worked fine.



    Alright, this is great news! It is a sign that very soon you'll join our private SCJD champions club!
     
    Olu Shiyan
    Ranch Hand
    Posts: 57
    Oracle Java
    • Mark post as helpful
    • send pies
      Number of slices to send:
      Optional 'thank-you' note:
    • Quote
    • Report post to moderator
    Roberto,

    Yup, hopefully.



    Cheers
     
    Amateurs built google. Professionals built the titanic. We can't find the guy that built this tiny ad:
    Smokeless wood heat with a rocket mass heater
    https://woodheat.net
    reply
      Bookmark Topic Watch Topic
    • New Topic