OutOfMemory when using CeWolf and JFreeChart

[Ole V. Villumsen]

In a smaller web application I added two timeseries charts using CeWolf 1.1 and JFreeChart 1.0.13. When we deployed the application, it ran for an hour, then started reporting lots of OutOfMemoryError.

I'm convinced that the problem has to do with the fact that our JSP page automatically refreshes every 5 seconds, which causes the charts to be redrawn. The charts themselves only change once every 5 minutes, but there is information in the page that we need to update within 5 seconds when it changes. I know that the long term solution i AJAX, we will come around to that later this summer.

Still, I had not expected this behaviour, and we would like our application to run 24*7, which it did nicely before. Does anyone know of a memory leak in either CeWolf or JFreeChart? Maybe even of a way to circumvent it? I just wanted to ask before I acquire a profiler.

Our application runs in a Tomcat server. There are 12 data points in each plot. I use two chart postprocessors for each chart, one de.laures.cewolf.cpp.TitleEnhancer and a homegrown one that fixes the range of the y scale (range scale) to be from 0 to 0.25 regardless of the data. For the time being, we have very few users, say 2 or 3, but that is expected to grow.

Any hints appreciated.

[Ulf Dittmer]

If you suspect that the chart is being created too often, make sure you're implementing the DatasetProducer.hasExpired method correctly. Of course, if there is an actual memory leak, then this will only postpone the OOME, not prevent it..

[Ole V. Villumsen]

Thank you very much, Ulf, for the reply.

It's more or less a separate issue, but I'm confused about the hasExpired method. The Javadoc says "If the data which had already been used for chart rendering is still valid this method should return true." This seems to indicate the opposite of what the method name says. The description of the return value seems to agree with the name: "true if the data which had been produced with the passed in parameters has expired since its creation, false otherwise". Can you help? I have tried returning false always and I have tried returning true always, but as far as I can tell from my log files, the chart is produced anew each time (every 5 seconds) in both cases. I can also reproduce the OutOfMemoryError with both versions. Only it took a little longer with the version that returns true.

[Ole V. Villumsen]

From jmap and jhat I gather that my Tomcat Catalina session (an org.apache.catalina.session.StandardSession object) through its attributes holds on to thousands of de.laures.cewolf.taglib.SimpleChartDefinition objects. I suspect that it generates a new one for each page refresh and doesn't let go of the obsolete ones.

I shall try and see what happens if I close and restart my web browser regularly while the application is running. Problem will remain, though, since my boss would like to be able to have our web page open at alle times, even display it on a large screen at the company entrance.

I should very much like to hear from others that have tried using dynamic CeWolf charts in a web application that's supposed to run continuously with minimal downtime. Hear what memory consumptions you have experienced.

[Ulf Dittmer]

Try adding a removeAfterRender="true" attribute to the <img> tag. That should cause the chart definition to be removed after the image has been rendered.

[Ole V. Villumsen]

Thanks again! I noticed that attribute too, feel foolish I didn't see ot from the outset. Anyway, it helps noticeably, but my problem doesn't seem to be all gone. I will report back.

[Ole V. Villumsen]

I still have very many ChartImageDefinition objects on the heap long after I've closed down my browser. Wonder if we should implement our own CeWolf storage instead of TransientSessionStorage.

I tried putting <%@ page session="false" %> on the JSP page, but that causes de.laures.cewolf.storage.AbstractSessionStorage to throw a NullPointerException, and the page display breaks.

Since we expect long sessions, we need to find a solution where obsolete charts are thrown away during the session rather than after it ends.

[Ulf Dittmer]

Are you using an <imgurl> tag, by any chance? I'm just looking at the session storage code, and there seems to be a leak associated with those.

[Ole V. Villumsen]

No such luck; no imgurl tag here. Thanks for looking.

[Ulf Dittmer]

Any <legend> tag would also need the removeAfterRender, by the way.

[Ole V. Villumsen]

On the contrary I have showlegend = "false" on both my charts.

Should I use the usecache attribute and what value should I give it?

<table width="540">
	<tbody>
		<jsp:useBean id="timeDataProducer"
			class="dk.certifikat.oces2.monitoring.dashboard.charts.TimeSeriesDataProducer" />
		<jsp:useBean id="timeProcessor"
			class="dk.certifikat.oces2.monitoring.dashboard.charts.TimeSeriesChartPostProcessor" />
		<jsp:useBean id="titleSizer"
			class="de.laures.cewolf.cpp.TitleEnhancer" />
		<tr>
			<th>...</th>
			...
		</tr>
		<tr>
			<td>NemID Log-in</td>
			<td align="center">
				<img border="0" width="33" height="33"
					src="/export/system/modules/dk.nemid.oc/resources/images/${ibss_login_state}" />
			</td>
			<td>...</td>
			<td rowspan = "4">
				<cewolf:chart id = "loginChart"
						title = "Login-svartider"
						type = "timeseries"
						showlegend = "false">
				    <cewolf:data>
				        <cewolf:producer id="timeDataProducer">
				        	<cewolf:param name="monitorTaskName"
				        			value="IBSS 1-factor loginrobot monitor" />
				        </cewolf:producer>
				    </cewolf:data>
				    <cewolf:chartpostprocessor id="timeProcessor">
						<cewolf:param name="maxYMillis" value="250" />   
				    </cewolf:chartpostprocessor>
				    <cewolf:chartpostprocessor id="titleSizer">
						<cewolf:param name="fontsize" value="11" />
					</cewolf:chartpostprocessor> 
				</cewolf:chart>
				<cewolf:img chartid="loginChart"
					renderer="cewolf" width="256"
					height="160" removeAfterRender="true" />
			</td>
		</tr>
		...

[Ulf Dittmer]

No, a <legend> tag creates a separate image for the legend - in case you want the legend somewhere other on the page rather than right in the image. But you're not using it, so all is good.

I don't think fiddling with useCache will help - it's about reusing (and caching) data, not images.

One more thing to try is to use LongTermSessionStorage instead of TransientSessionStorage (in web.xml). It purports to remove objects from the session automatically after a certain time; I say "it purports", because I've never tried it myself.

[Ole V. Villumsen]

Thank you very much for the suggestion!

When checking the web.xml syntax for specifying a storage in the CeWolf tutorial (on http://cewolf.sourceforge.net/), I also stumpled across the session-timeout setting (1 minute), and copied it. That alone helped much, memory consumption built up much more slowly. Seems I went too quickly through the tutorial the first time. However, I still had thousands of ChartImageDefinition objects on the heap, referenced from a large ConcurrentHashMap in the session with String keys representing integer values, like "-1237369985".

However, additionally specifying LongTermSessionStorage in web.xml and a timeout of 10 (seconds) in each cewolf:img tag on my JSP page seems to have eliminated the leak completely. That's so great!

[Ulf Dittmer]

Glad to hear that it worked out well.