Granny's Programming Pearls
"inside of every large program is a small program struggling to get out"
JavaRanch.com/granny.jsp
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Why Velocity Templates takes larger memory JIRA

 
coumaravel Cattavarayane
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

We are using the velocity templates extensively to
generate HTML reports. Normally our templates size on
the Disk is around 20-50kb, when it gets loaded into
Application(Tomcat) memory it turns to 3-5MB.

As of now we have 500 templates, in near future we are
planning to increase this to 1500.

Any Idea why this is happening. Any help in this is
much appreciated, as this is becoming a bottleneck for
the scalability of our application.

Thanks in advance.

Regards,
Coumar
 
Peter Chase
Ranch Hand
Posts: 1970
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Velocity templates get parsed and converted into in-memory data structures. The in-memory data structure can be larger than the original file.

One reason is that static text in the file ends up as UTF-16 Java strings, but most files use some form of 8-bit encoding where the average size of a character is about halved compared to UTF-16.

On the other hand, comments in the Velocity templates do not end up in the in-memory data structures.

I would not expect such a large difference between the size of the disk file and the size of the in-memory data structure. How are you measuring the size in memory? It is notoriously difficult to measure memory consumption in Java, or other garbage-collected languages. You can't just look at the VM size, because that typically includes a whole load of dead objects, waiting to be GCd.

A final tip, if you are using Windows Task Manager to estimate memory use. The default setting of Task Manager shows a column "Memory Usage", but this is the physical RAM usage, which is of little use. What you want is the column "VM Size", which is not shown by default, but can be turned on.
 
coumaravel Cattavarayane
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

Thanks for your reply.

All the velocity templates are getting loaded into the application memory on the startup. We have enabled the GC logs to find out the actual memory usage within the application.
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13071
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If memory is still a problem you might investigate a caching scheme so that frequently used templates stay in the memory cache but infrequently used ones get dropped. There are many open source caching toolkits, the whirlycache project seems active.
Bill
 
Peter Chase
Ranch Hand
Posts: 1970
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by coumaravel Cattavarayane:
We have enabled the GC logs to find out the actual memory usage within the application.


Are you sure that gives you the information you want? JavaRanch has hosted several discussions about how to find out the size of Java objects, and I don't remember seeing anyone say you could find it out from the GC logs.
 
coumaravel Cattavarayane
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We found the actual memory taken by the Velocity templates by the following approach:

1. Start the application without loading any templates - note down the GC details
2. Start the application with loading templates in the startup - note down the GC details.

By doing this, we can find out how much memory is being used for the Velocity templates.
 
Ilja Preuss
author
Sheriff
Posts: 14112
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by coumaravel Cattavarayane:
We found the actual memory taken by the Velocity templates by the following approach:

1. Start the application without loading any templates - note down the GC details
2. Start the application with loading templates in the startup - note down the GC details.

By doing this, we can find out how much memory is being used for the Velocity templates.


Ah, yes - but not only by the templates themselves, but presumedly also by all the code that is needed to handle the templates. Remember, those classes are only loaded when actually needed.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic