Forums Register Login

FOP - Out of memory

+Pie Number of slices to send: Send
Hi.

I am trying to use FOP to get an e-mail attachment returned as byte array.

It seems that I am running out of memory when I create the Fop object:
Fop fop = fopFactory.newFop(MimeConstants.MIME_PDF, foUserAgent, out);

I am pretty sure it has something to do with the ByteArrayOutputStream.

Any help would be appreciated?



Server log:
+Pie Number of slices to send: Send
How big is the attachment? How much memory does the servlet container have allocated? FOP needs a lot.
+Pie Number of slices to send: Send
The file is only 12 kb.

The server have 2560 MB and Xmx512m.

I have tried to save the attachment as a file on disk.

The error now come here:
transformer.transform( src, result );



Server log:
+Pie Number of slices to send: Send
Oh yuck. PermGen space.

PermGen memory isn't the same thing as regular heap memory. For Tomcat, the default PermGen space is "only" 128MB, but if you're using an ORM or other systems that do a lot of static objects, it's often advisable to double that.

If you are running Tomcat, search the Tomcat forum for more on this particular issue.
+Pie Number of slices to send: Send
I find it worrying that this 'out of memory' is happening inside a Servlet. If the Servlet handles many simultaneous requests then it is possible that two or more FOP transformation will be active at the same time and if you are short of memory with just one request then, obviously, you are going to be worse off with two or more simultaneous requests. You may have to consider having a process, maybe external to the Servlet container, which queues FOP transformations to make sure that you limit the number of active transformations. Of course this could mean that your average response time will increase but that must be better than an 'out of memory' condition.
+Pie Number of slices to send: Send
 

Richard Tookey wrote:I find it worrying that this 'out of memory' is happening inside a Servlet. If the Servlet handles many simultaneous requests then it is possible that two or more FOP transformation will be active at the same time and if you are short of memory with just one request then, obviously, you are going to be worse off with two or more simultaneous requests. You may have to consider having a process, maybe external to the Servlet container, which queues FOP transformations to make sure that you limit the number of active transformations. Of course this could mean that your average response time will increase but that must be better than an 'out of memory' condition.



Depending on the size and complexity of the request, running FOP inside a servlet is not an unrealistic thing to do. If you want a web request to generate a PDF via FOP, ultimately, it's going to be a servlet receiving the request anyway. The apache Cocoon webapp framework relies on FOP for its PDF functionality, for example.

The issue here isn't "running out of memory", it's running out of PermGen memory. PermGen space isn't the same thing as general space. It's allocated differently, and, as I mentioned before, it's not as big as the regular heap memory pool. Nor can adjusting the heap limits help, since there's a completely separate parameter that sets the PermGen size.
+Pie Number of slices to send: Send
 

Tim Holloway wrote:
The issue here isn't "running out of memory", it's running out of PermGen memory. PermGen space isn't the same thing as general space. It's allocated differently, and, as I mentioned before, it's not as big as the regular heap memory pool. Nor can adjusting the heap limits help, since there's a completely separate parameter that sets the PermGen size.



OK it is not running out of general memory but 'PermSpace' but surely this doesn't change the fact that FOP is consuming a too much of the container's resources for just one request and simultaneous requests will make it far worse. Many year ago when I first used FOP in earnest the FOP transformation took so much memory (JDK1.4 so not PermGen involved) that the FOP processing was done in a separate computer.
+Pie Number of slices to send: Send
Actually, we have no idea how much general heap memory this particular transformation is using. All we truly know is that it wants more PermGen space. And lacking actual breakdowns, I can't even guarantee that FOP is what's eating PermGen. I've never had FOP blow out that particular resource, although ORM-based apps are a perennial nuisance in that regard for me. So FOP+Hibernate could be more a Hibernate problem than a FOP problem, as an example.

I would definitely monitor system resource usage, however. There may be FOP-related issues. Although a "lot" of memory is something whose absolute size changes from year to year. Still, in order to measure how much heap memory is being eaten - and whether it's worth the expense to redesign for an external processor - first we need to resolve the immediate problem. That can be done with a simple VM setting, which is a lot faster and easier than re-architecting the app for gains which may or may not materialize.
+Pie Number of slices to send: Send
 

Tim Holloway wrote:, which is a lot faster and easier than re-architecting the app for gains which may or may not materialize.



I have voiced my concerns about FOP memory resource usage and have not suggested a "re-architecting". Having worked on a project that employed an IBM high volume web site consultant I know that establishing a client usage pattern is paramount in deciding on an architecture. Knowing the usage pattern allows one to model various architectures to establish resources required. My experience is that FOP is (or at least was) a very resource hungry process and may be bad news when used in a web site. I know little about the more recent garbage collectors and bow to your experience but I wanted the OP to be aware of possible resource shortage problems. I don't want to get into an argument over this.
+Pie Number of slices to send: Send
Thanks for your comments.

We solved our memory issue by simply add:

-XX:+CMSClassUnloadingEnabled
-XX:+CMSPermGenSweepingEnabled
-XX:MaxPermSize=256m

For some reasons that we don't know, the MaxPermSize was reseted and too low.

When this is said our traffic is still low so I am not sure about your discussion.

I will keep in mind about the simultaneous active transformations...
+Pie Number of slices to send: Send
MaxPermSize defaults to 128m. 256m will almost always do if you need more, although I did have to do a bump up to 384m on a complex system this morning.

Fortunately, PermGen space is not used for most memory needs and it's pretty much all shared memory, so adding more users is unlikely to require massive amounts of PermGen.
Looky! I'm being abducted by space aliens! Me and this tiny ad!
a bit of art, as a gift, that will fit in a stocking
https://gardener-gift.com


reply
reply
This thread has been viewed 4310 times.
Similar Threads
Java : avalon framework configuration exception
XSL-FO Multiple Page Sequences
Corrupt PDF file while using .fo to .PDF using FOP
ServletOutputStream.write not working in IE 7
Problem in ServletOutputStream
More...

All times above are in ranch (not your local) time.
The current ranch time is
Apr 16, 2024 04:01:44.