• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Need your expert opinion!

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Anyone please help?
I am in a debate with the vendor about the following. In a web application, they convert all input into uppercase using String toUpperCase. Their reasoning is it helps with quering the data in a search (test vs Test). I suggest storing data as entered (filtering special characters, etc) and use SQL Server upper/lower function instead. Eg, for search criteria last=t*, query would be:
select * from user where upper(last) like upper('t%').
A small query analysis does show a performance decrease with using the function vs
select * from user where last like('T%'), given all data is CAPITALIZED

So, what is more memory expensive, given about 20 simultaneous users, calling toUpperCase when passing values into a database, or described above?
 
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by A Len:
So, what is more memory expensive, given about 20 simultaneous users, calling toUpperCase when passing values into a database, or described above?


Your database may have the concept of a function-based index. In Oracle, you can index upper(column) to improve performance. To query you do

That would solve the performance issues you're seeing. If that's not an option, it means doing the search will result in a full table scan. For 50 rows, no big deal. For 1 million users, eek!
As to the wasted memory of uppercasing the strings, 1) they're probably short, 2) String doesn't take that much memory, and 3) the objects are short-lived and the JVM can optimize their use pretty well. I wouldn't worry about it; not when you're talking about wasting a few kibibytes here and there.
 
ALeX
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
appreciate your response!
I haven't seen the code yet, so there are many blanks. Database is MS SQL2000 Enterprise, running on a multi-processor server. Web app is Struts-based, with Tomcat 4.1.29 running on another multiprocessor box. Both are Windows 2000 Server. I am trying to troubleshoot the outOfMemory problems. Before spending too much time with JVM performance tuning, I want to make sure all other 'more obvious' issues are taken care of.
So, for the following scenario:
For 10 new records submitted by multiple users, containing about 30 fields each, 20 are strings that are capitalized, there are no memory problems? If these operations are indeed short-lived and do not affect JVM's performance, then I'll look for other problems.
 
David Harkness
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by ALeX:
For 10 new records submitted by multiple users, containing about 30 fields each, 20 are strings that are capitalized, there are no memory problems? If these operations are indeed short-lived and do not affect JVM's performance, then I'll look for other problems.


Let's assume the average string size for those fields is 40 characters. So with 20 string fields per object, a base size of 12 bytes for a String, and 2 bytes per character, you're using about 1840 bytes for temporary uppercase versions of those fields per object. Of course, the non-uppercased fields that came from the web layer are also temporary and of the same size.
This isn't much memory, and it's happening all the time. If your JVM is recent, it should have an improved GC that deals with temporary object creation more robustly. I suspect there's a bigger problem somewhere, whether you're caching too many objects or keeping references around that should be nulled.
 
Bartender
Posts: 9626
16
Mac OS X Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
And for pete's sake invoke close() on your ResultSet, Statement and Connection objects (or return Connections to your connection pool). Probably the greatest single cause of memory leaks in web apps.
 
ALeX
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you all for a valuable feedback.
FYI, I found out something that I actually forgot. When creating a database in SQL Server, you can specify case-incensitive or case-sensitive search using collation (by default, it is set to case-incensitive).
For Doe, John search query
select * from user where lname like ('d%') and fname like ('j%')
returns the same results as
select * from user where lname like ('D%') and fname like ('J%')
 
ALeX
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by David Harkness:

Let's assume the average string size for those fields is 40 characters. So with 20 string fields per object, a base size of 12 bytes for a String, and 2 bytes per character, you're using about 1840 bytes for temporary uppercase versions of those fields per object. Of course, the non-uppercased fields that came from the web layer are also temporary and of the same size.
This isn't much memory, and it's happening all the time. If your JVM is recent, it should have an improved GC that deals with temporary object creation more robustly. I suspect there's a bigger problem somewhere, whether you're caching too many objects or keeping references around that should be nulled.


I do not have the source code for the server-side. I can only analyse the results, unfortunately, at this time. My main concern was with java.lang.outOfMemory exception. Webapp is running on Tomcat4.1.29(jdk1.4.1_02)/Struts1.1 on Windows 2000 Server, on a powerful multiprocessor server with 4Gb memory. I have been tuning JVM's GC options, but without the source code I can mostly guess where the problems are. Next week, I will load JProbe - hope it will show which classes stress CPU the most.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic