Win a copy of The Little Book of Impediments (e-book only) this week in the Agile and Other Processes forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

how to handle 10,000 records from database

 
Katerina Karen
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
in my web application I write a query that fetches 10,000 records from a remote database.how do i handele it in my web application for optimal performance
 
Andres Gonzalez
Ranch Hand
Posts: 1561
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There's a pattern called page-by-page iterator. It describes a solution to a problem like this one.
have a look at this link.
 
Steve Agarwal
Ranch Hand
Posts: 51
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I knew only about View list handler pattern of CORE J2EE.
Steve
 
Jim Doyle
Ranch Hand
Posts: 36
IBM DB2 Netbeans IDE Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It depends. You really need to understand more about databases and
database connectivity before you can answer questions about relative
performance.
Things you need to understand:
1. How is the result set delivered to JVM. In some cases, all 10,000
rows will be transmitted to the JVM. In other cases, blocks of perhaps
4K or 8K at are time are fetched as you iterate through the result set.
This has an impact on local memory utilization.
2. How is the statement prepared and at what isolation level. If
you are reading from the database at any level other than Uncommitted
Read, you are holding down some locks on the database. If you hold the
result set open too long, you may be affecting the performance of other
users of the database as they will busy wait on your row locks, or,
timeout because they reached the waiting limit to acquire row and table
locks. If you are reading from the database as Repeatable Read or Read
Stability, your front end application could adversely affect database
concurrency.
3. Use a scrolling cursor through the result set... Generally, folks
that display large amounts of data like this create alot of heap fragmentation from all of the string assembly that goes on... i.e. all
those TD and TR tags. You can design (or reuse) a class that is artfully
designed to construct HTML table elements from a result set cursor in a
way that is more memory friendly than simply string concatenation. A
certain app server I once used employed this technique - also known as
"scatter-gather" buffers from the Unix OS world, nothing is new under
the sun.
-- Jim (SCJP2, IBM DB2 Associate, Unemployed)
 
Michael Ernest
High Plains Drifter
Sheriff
Posts: 7292
Netbeans IDE VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This topic is closed until the topic starter changes his/her display name to match our naming policy.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic