Win a copy of Practical SVG this week in the HTML/CSS/JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

From Memory or from DB?

 
Andres Delrotti
Ranch Hand
Posts: 178
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello members of the Javaranch community.

I have this problem. Currently we have an ASP reporting application. It updates the information it shows by calling databse stored procedures on a specific time interval. We are now planning of converting this application into a Java application. My boss suggested to me that since the information that we retrieve from the database does not update that often, maybe we could just store these information in our application's memory. The initial plan was to have a cron job run every 5 am in the morning which would query in database and transfer it in our application's memory. We would then just have a listener in our app which connects to a socket from an external system to detect changes in stored data and just update it our memory. Our "memory" would be composed of Java collection classes such as Maps and Lists which stores records in the database and we are reading about 10 to 20,000 records each day. We would retrieve all these records for a specific day and just do "lazy caching" for specific user queries.

I had feedback from some of my colleagues and they say that having it implemented in the memory is much too risky. A lot of them did the same thing before and encountered a lot of outOfMemory exceptions. On the other hand, having our application run complicated queries from the database for short periods of time wherein the actual data does not update very often may also not be a good idea. It could take up resources and downgrade performance of other applications connecting to the same database.

I hope you people can help me to decide on this if ever you've been in this dilemma before. I also want to know what KEY factors should I consider in making my decisions...or maybe suggest to me some Commercial profiling tools I could use to measure performance from both choices.

What would you suggest? I'd appreciate any form of help. Thank you very much.
 
steve souza
Ranch Hand
Posts: 862
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
One of my pet peaves is when developers cache without knowing what the noncached performance is. Caches are a pain to manage as it is often difficult to know when the cache should be updated. There are caching products out there should you need them, but still I would say the first step is to monitor your code.

The current version of the open source JAMon can monitor sql - You can look at hits/avg/min/max and more for each query, as well as look at information about the most recent N queries, and see any exceptions they have thrown. This is all viewable via a web app that comes with jamon or you can retrieve the raw data for display.

The current version requires you to wrap your connection with the JAMon monitor. The next release will remove this restriction and allow you to monitor any jdbc driver by wrapping it in the jamon proxy jdbc driver. This approach will require no code changes, but for now...




That is all it takes to get a wealth of statistics on jdbc, and the sql it executes. See the demo and links below for more information.
 
Andres Delrotti
Ranch Hand
Posts: 178
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
^^
Thanks for the help. Is there any way to measure cached performance? what specific memory attributes should I specifically monitor to have out of Memory Exceptions?
 
steve souza
Ranch Hand
Posts: 862
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm not sure I understand your questions, but if you are asking how to use jamon to monitor a cache you create then I would create an interface and then do the following:



All public methods of 'MyCacheInterface' will be monitored. Also all exceptions thrown by this interface will be monitored.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!