Does the persistence layer know when the database has changed?

[Tim Sparg]

Hi all

I'm investigating the use of JPA for a rewrite of our main application.

We have a main application that our clients use, and then multiple utility applications that read data from the database and send it of, or receive data and write it to the database.

If our main application is using a persistence layer and a utility writes data directly to the database does the persistence layer know that it needs to get the new data?
If not how would one go about telling it to refresh specific data?

cheers
Tim

[Arun Kumarr]

If our main application is using a persistence layer and a utility writes data directly to the database does the persistence layer know that it needs to get the new data?
If not how would one go about telling it to refresh specific data?

-- No, It wouldn't. Peristence layers however would be mature enough to do a select-before-update, automatic version checking and report a stale state exception to your application
which you can handle elegantly.

[Tim Sparg]

Arun Kumarr wrote:
-- No, It wouldn't. Peristence layers however would be mature enough to do a select-before-update, automatic version checking and report a stale state exception to your application
which you can handle elegantly.

That works for data that is being updated, however I"m more thinking of static data, or data that doesn't change often.
ie Exchange rates - these would be changed at most once a day, and never updated from the application - In this case you would read stale data until the persistence layer was restarted?

Would it be possible for a trigger to interact with the persistence layer, notifying it of a change?

thanks
Tim

[Arun Kumarr]

For static data we should be using cache managers along with persistence manager, which can refresh data on a periodic basis.

[Ran Pleasant]

Tim Sparg wrote:Hi all

I'm investigating the use of JPA for a rewrite of our main application.

We have a main application that our clients use, and then multiple utility applications that read data from the database and send it of, or receive data and write it to the database.

If our main application is using a persistence layer and a utility writes data directly to the database does the persistence layer know that it needs to get the new data?
If not how would one go about telling it to refresh specific data?

Tim

About 8 or 10 years back I worked on a Swing application that displayed live stock trades for analysts of a company with a seat on the New York Stock Exchange, those analysts needed the freshest data possible. The primary model of my swing application was updated by a thread that listened constantly to a socked connection to an application written in C++. My understanding was that the C++ application was connected to some type of Oracle queue. When new data was entered into a specific table on the Oracle database it would post a message on the queue. The C++ application would then read the new data and push it up the socket to my application. Please note that I never saw the C++ application and really don't know how it was connected to Oracle. However, I do know that when new stock information hit the Oracle database a couple of seconds later it would appear in my Swing applications.

If you are using Oracle you might talk to your DBA to see how all of that might work for you.

[Tim Sparg]

Ran Pleasant wrote:
About 8 or 10 years back I worked on a Swing application that displayed live stock trades for analysts of a company with a seat on the New York Stock Exchange, those analysts needed the freshest data possible. The primary model of my swing application was updated by a thread that listened constantly to a socked connection to an application written in C++. My understanding was that the C++ application was connected to some type of Oracle queue. When new data was entered into a specific table on the Oracle database it would post a message on the queue. The C++ application would then read the new data and push it up the socket to my application. Please note that I never saw the C++ application and really don't know how it was connected to Oracle. However, I do know that when new stock information hit the Oracle database a couple of seconds later it would appear in my Swing applications.

If you are using Oracle you might talk to your DBA to see how all of that might work for you.

Hi Ran

We're using MSSQL for our sins ;)
We've been thinking of writing something similar with a combination of triggers and assemblies.

You know I've been looking at JPA for awhile now trying to figure out how to get it to do what I need... and I unfortunately think that it doesn't work for my current project for a couple of reasons

JPA seems to need exclusive access to the database - this is just not a reality for us, as we have several applications that directly insert data into the database.

We need fresh data - Following on from the above there is no easy way to get fresh data from the database.

Entity generation doesn't work - Automated entity creation either hangs (I've tried Netbeans,Eclipse,Jboss Developer) or creates an invalid database schema. I would rather create all of those entities myself than actually have to go and check that they are correct.

Views are second class citizens - We use views everywhere, and in order to get views to work correctly you need to modify the entity thats generated.

Read from view update to table - We often read from views and then update back to the table, again there doesn't seem to be an easy way to do this.

Entity joins are a bit of a misnomer - This is perhaps a bit less concrete than the others but just as important. Entites are created with joins to other tables/entites. This works well if you are doing all of your work on the the server/JPA tier however the moment that you have an entity that is remote from that tier and try to follow a non eager join then everything falls apart. (blob is a perfect example)

What I'm thinking of doing currently is using eodsql as a a very simple starting point to creating a my data access layer. Quite a departure from JPA which tries to do everything for you!

cheers
Tim

[Craig De La Hunt]

Hi Tim

I have given my own opinion on your questions below and how I would tackle them. I will mail you my test case so that you can reference it. Also I found this wikibook, it was really useful and I cleared a lot of mud from the water. I have also referenced it below with regards to caching.
http://en.wikibooks.org/wiki/Java_Persistence

• JPA seems to need exclusive access to the database - this is just not a reality for us, as we have several applications that directly insert data into the database.

 You can turn off L2 caching and only use L1 caching in you persistence.xml “<shared-cache-mode>NONE</shared-cache-mode>”. L1 cache, caches the objects state for the duration of the transaction. There are a few issues with flushing the data to the database which just require a bit of thought. From the article below – when you don’t use an extended persistence context - “Typically the JEE server will inject the application with a proxy to an EntityManager, and after each JTA transaction a new EntityManager will be created automatically or the EntityManager will be cleared, clearing the 1st level cache.”
I have tested this and it works as stated. http://en.wikibooks.org/wiki/Java_Persistence/Caching#Stale_Data

 Also, why do you allow other programs to connect directly to your application? Could they not rather call remote methods or make webservice calls to the application server where the persistence layer is, then perhaps you can use L2 cache again, that is unless you have a legacy system written in an old language that cannot use newer technologies.

• We need fresh data - Following on from the above there is no easy way to get fresh data from the database.

 Same as above, use L1 caching instead of L2 caching. Persistence.xml - <shared-cache-mode>NONE</shared-cache-mode>

• Entity generation doesn't work - Automated entity creation either hangs (I've tried Netbeans,Eclipse,Jboss Developer) or creates an invalid database schema. I would rather create all of those entities myself than actually have to go and check that they are correct.

 Ok, Ill agree with you here. When generating entities on a complex database maybe you should generate a single entity at a time or create them yourself and then do the OR Mapping yourself. I prefer creating the entites myself and using the drop and create tables strategy “<property name="eclipselink.ddl-generation" value="drop-and-create-tables"/>” to check the validity of the mapping compared to my database. The more often you do this the better you get at it and the less you have to check!

• Views are second class citizens - We use views everywhere, and in order to get views to work correctly you need to modify the entity thats generated.

 Create the entity yourself.
Take a look at this. http://www.adam-bien.com/roller/abien/entry/mapping_jpa_entities_on_view

• Read from view update to table - We often read from views and then update back to the table, again there doesn't seem to be an easy way to do this.

 Here you could read the view into a read only entity and in the update method you could create and instantiate a new entity object of the table you wish to update and assign the variables from the view object to the new entity object, and then update the table. Not all views are updatable

• Entity joins are a bit of a misnomer - This is perhaps a bit less concrete than the others but just as important. Entites are created with joins to other tables/entites. This works well if you are doing all of your work on the the server/JPA tier however the moment that you have an entity that is remote from that tier and try to follow a non eager join then everything falls apart. (blob is a perfect example)

 Well how would you do it with JDBC?
You should be calling a method (fetchBlob) on a stateless session bean that populates the blob for you.
Think that would work best for blobs and then you could use fetch joins for all other lazy objects if you need it. Also, while we on the subject of joins, you could just use fetch joins and batch joins in your sql and then you may no longer need views.