i have a problem with jdbc. i have a DB with something like 20,000 records , (and change :-)) , i'm exporting them into xml file, my problem is that when i retrieve then from the DB the first 100 is fast , but then it takes me something like hour to getall the rows, i can open threads but it still a lot of time, does any one have an idea?
Jonathan, Have you done any timing to verify that the DB access is your bottleneck? What DB engine are you using? What JDBC driver implementation are you using? I can't believe it's that slow (not that I'm doubting you - which I am. ) From my experience, I would think creating the XML string for the 20K+ records is your bottleneck rather than the DB fetches. What technique are you using to fetch 100 records at a time? I would recommend commenting out the XML creation process, and see how long it takes to just fetch all those records. For example, you can do this.
[This message has been edited by Peter Tran (edited January 21, 2001).]
although i forgot to mention , i'm using JDBC -ODBC bridge , and Oracle7 . i'll make the test, and get back to you, lets assume that's not what's keeping me behind. how can i still retrived 20k rows ? i don't mind doing the XML after i get the rows , but i hv no idea where to keep them.. i can always hold them in a vector , or somthing like that, but i think it would take me even longer. thx for the help Jonathan
Ugh...Using the JDBC-ODBC bridge might be another one of your problem. Read this topic regarding what we think of the JDBC-ODBC bridge. Notice Thomas' comments. If there are memory leaks in the implementation, then it may explain your performance problem. Try running your application with the -verbosegc and see if your memory usage creeps up. -Peter
Jonathan, Your retrieved 5210 records in less than 1/2 second (441 milliseconds). That's not bad for the JDBC-ODBC bridge. I was expecting it to be much slower. Working the numbers, 20K+ records should only take you 2 seconds to retrieve. Assuming that the database access is acceptable, then building the XML string will be your bottleneck. How are you doing this now? Don't be bashful, and show us some code my man... -Peter PS. Please read the FAQ to learn how to format your code so it's readable. [This message has been edited by Peter Tran (edited January 21, 2001).]
Yikes!!! Horror of horrors...No wonder your program takes so long. Your program is creating a bazillion String objects. You should use a StringBuffer object and use the StringBuffer.append() method to create your XML string. Outside the loop, do StringBuffer.toString() to get a String object. Oh man, the GC must really hate running your application. Before your while statement where you allocate a new String object, replace that with a StringBuffer buf = new StringBuffer(); Everywhere inside your code where you have "xml = xml +" replace that with buf.append(). Outside, the while loop execute String xmlString = buf.toString(); to get your XML string object. Make this change and your program will run like the RoadRunner..."beep...beep..." -Peter
thank :-) thank :-) thank :-) thank :-) thank :-) thank :-) u've just saved me hours of work.. it always happen to me , i never knew there's a StringBuffer object, but thanks to you i know... i'll give it a try and get back to you with the time it took me. jonathan