XSL-FO Multiple Page Sequences

[Kamohelo Mofokeng]

Hi All,

I'm trying to implement a solution to create multiple page sequences (from the thread: https://coderanch.com/t/125494/XML/Mulitple-fo-page-sequence) in my xsl-fo file. I get a java.lang.OutOfMemoryError when I transform large data.

Although I increased -Xms/-Xmx & java_arguments to 2G, the problem still persists.

Here's how my xsl file looks:

<xsl: template match="/">   
      <fo:root xmlns:fo="http://www.w3.org/1999/XSL/Format">   
        <xsl:call-template name="LAYOUT"/>   
        
        <xsl:choose>   
          <xsl:when test="ROOT/MEM-STMT/COUNTRY ='MU'">     
            <fo:page-sequence master-reference="pre-print">   
              <fo:static-content   
                  flow-name="xsl-region-before-first">   
                <fo:block>   
                  <xsl:call-template name="HEADER"/>   
                </fo:block>   
              </fo:static-content>   
                 
              <fo:static-content   
                  flow-name="xsl-region-after">   
                <fo:block text-align="center">   
                  <xsl:call-template name="FOOT"/>   
                </fo:block>   
              </fo:static-content>   
        
              <fo:flow flow-name="xsl-region-body">   
                <fo:block>   
                  <xsl:choose>   
                    <xsl:when test="ROOT/MEM-STMT/FIN-BAL/FIN-BAMT !='0'">   
                      <fo:block>   
                        <xsl:call-template name="FIN-BAL"/>   
                      </fo:block>   
                    </xsl:when>   
                  </xsl:choose>   
                  <xsl:apply-templates select="PROVIDER"/>   
                  <xsl:apply-templates select="MEM-TOTALS"/>   
                  <xsl:apply-templates select="F-OTHER"/>   
                  <xsl:apply-templates select="DUE"/>   
                  <xsl:apply-templates select="REASON"/>   
                </fo:block>   
              </fo:flow>   
            </fo:page-sequence>   
          </xsl:when>   
          <xsl:otherwise>   
            <fo:page-sequence master-reference="global">   
              <fo:static-content   
                  flow-name="xsl-region-before-first">   
                <fo:block>   
                  <xsl:call-template name="HEADER"/>   
                </fo:block>   
              </fo:static-content>   
                 
              <fo:static-content   
                  flow-name="xsl-region-after">   
                <fo:block text-align="center">   
                  <xsl:call-template name="FOOT"/>   
                </fo:block>   
              </fo:static-content>   
                 
              <fo:flow flow-name="xsl-region-body">   
                <fo:block>   
                  <xsl:choose>   
                    <xsl:when test="ROOT/MEM-STMT/FIN-BAL/FIN-BAMT !='0'">   
                      <fo:block>   
                        <xsl:apply-templates select="FIN-BAL"/>   
                      </fo:block>   
                    </xsl:when>   
                  </xsl:choose>       
                  <xsl:if test="ROOT/MEM-STMT/ACC/PROV[position() mod 30 = 1]">   
                          <fo:block>    
                                   <xsl:call-template name="PROVIDER"/>   
                          </fo:block>                     
                  </xsl:if>                     
                  <xsl:apply-templates select="MEM-TOTALS"/>   
                  <xsl:apply-templates select="F-OTHER"/>   
                  <xsl:apply-templates select="DUE"/>   
                  <xsl:apply-templates select="REASON"/>                                         
                </fo:block>   
              </fo:flow>   
            </fo:page-sequence>      
          </xsl:otherwise>   
        </xsl:choose>                 
      </fo:root>   
    </xsl:template>

The PROVIDER template returns a lot of data & this is where FOP runs out of memory.

& the input xml file looks as follows:

<ROOT>   
      <MEM-STMT>   
        <COUNTRY>co</COUNTRY>   
        <CURRENCY>cu</CURRENCY>   
        <STMT>   
          <ST-NO>1</ST-NO>   
          <ST-DATE>21-JUL-11</ST-DATE>   
          <CC>21</CC>   
          <YY>11</YY>   
          <MM>07</MM>   
          <DD>21</DD>   
        </STMT>   
        <MEM-DET>   
          <MEM-NO>9</MEM-NO>   
          <MEM-PROD-LEV>24</MEM-PROD-LEV>   
          <MEM-OPTION>option</MEM-OPTION>   
          <MEM-EMP-NO/>   
          <MEM-EMP-NAME>name</MEM-EMP-NAME>   
        </MEM-DET>   
        <MEM-ADDR>   
          <MEM-NAME>name</MEM-NAME>   
          <MEM-ADDR1>                  Addr1</MEM-ADDR1>   
          <MEM-ADDR2>                           Addr2</MEM-ADDR2>   
          <MEM-ADDR3>                         Addr3</MEM-ADDR3>   
          <MEM-SUB>                            Sub</MEM-SUB>   
          <MEM-CITY>                          +2</MEM-CITY>   
          <MEM-REG>                   Employer:reg</MEM-REG>   
          <MEM-CNTRY/>   
          <MEM-PCODE/>   
          <MEM-EMPCODE/>   
        </MEM-ADDR>   
        <ACC>   
          <PROV>   
            <PR-NO>1</PR-NO>   
            <PR-NAME>  pr</PR-NAME>   
            <REF-NO>1</REF-NO>   
            <CLM>   
              <REF>1</REF>   
              <CL-DEP-NO>04</CL-DEP-NO>   
              <CL-DEP-NAME>dep</CL-DEP-NAME>   
              <CL-DOS>10-APR-11</CL-DOS>   
              <CC>21</CC>   
              <YY>11</YY>   
              <MM>04</MM>   
              <DD>10</DD>   
              <CL-TAR-DRUG-CD>1 </CL-TAR-DRUG-CD>   
              <CL-TAR-DRUG-DESC>desc </CL-TAR-DRUG-DESC>   
              <CL-NO>1</CL-NO>   
              <CL-LINE>2</CL-LINE>   
              <CL-AMT>16.8</CL-AMT>   
              <CL-TAR-AMT>16.8</CL-TAR-AMT>   
              <CL-PAID-PROV>16.8</CL-PAID-PROV>   
              <CL-PAID-MEM>0</CL-PAID-MEM>   
              <CL-RSN/>   
            </CLM>   
            <CLM>   
              <REF>1</REF>   
              <CL-DEP-NO>04</CL-DEP-NO>   
              <CL-DEP-NAME>dep </CL-DEP-NAME>   
              <CL-DOS>20-APR-11</CL-DOS>   
              <CC>21</CC>   
              <YY>11</YY>   
              <MM>04</MM>   
              <DD>20</DD>   
              <CL-TAR-DRUG-CD>1 </CL-TAR-DRUG-CD>   
              <CL-TAR-DRUG-DESC>desc </CL-TAR-DRUG-DESC>   
              <CL-NO>1</CL-NO>   
              <CL-LINE>2</CL-LINE>   
              <CL-AMT>50.4</CL-AMT>   
              <CL-TAR-AMT>50.4</CL-TAR-AMT>   
              <CL-PAID-PROV>50.4</CL-PAID-PROV>   
              <CL-PAID-MEM>0</CL-PAID-MEM>   
              <CL-RSN/>   
            </CLM>   
            <TOTALS>   
              <TOT-AMT>67.2</TOT-AMT>   
              <TOT-TAR-AMT>67.2</TOT-TAR-AMT>   
              <TOT-PAID-PROV>67.2</TOT-PAID-PROV>   
              <TOT-PAID-MEM>0</TOT-PAID-MEM>   
            </TOTALS>   
          </PROV>   
          <PROV>   
            <PR-NO>2</PR-NO>   
            <PR-NAME>  pr</PR-NAME>   
            <REF-NO>1</REF-NO>   
            <CLM>   
              <REF>1</REF>   
              <CL-DEP-NO>04</CL-DEP-NO>   
              <CL-DEP-NAME>dep</CL-DEP-NAME>   
              <CL-DOS>10-APR-11</CL-DOS>   
              <CC>21</CC>   
              <YY>11</YY>   
              <MM>04</MM>   
              <DD>10</DD>   
              <CL-TAR-DRUG-CD>1</CL-TAR-DRUG-CD>   
              <CL-TAR-DRUG-DESC>desc </CL-TAR-DRUG-DESC>   
              <CL-NO>1</CL-NO>   
              <CL-LINE>2</CL-LINE>   
              <CL-AMT>15</CL-AMT>   
              <CL-TAR-AMT>0</CL-TAR-AMT>   
              <CL-PAID-PROV>15</CL-PAID-PROV>   
              <CL-PAID-MEM>0</CL-PAID-MEM>   
              <CL-RSN/>   
            </CLM>   
            <TOTALS>   
              <TOT-AMT>15</TOT-AMT>   
              <TOT-TAR-AMT>0</TOT-TAR-AMT>   
              <TOT-PAID-PROV>15</TOT-PAID-PROV>   
              <TOT-PAID-MEM>0</TOT-PAID-MEM>   
            </TOTALS>   
          </PROV>   
          <PROV>   
            <PR-NO>1</PR-NO>   
            <PR-NAME>  pr</PR-NAME>   
            <REF-NO>I0428202</REF-NO>   
            <CLM>   
              <REF>I0428202</REF>   
              <CL-DEP-NO>03</CL-DEP-NO>   
              <CL-DEP-NAME>dep</CL-DEP-NAME>   
              <CL-DOS>10-APR-11</CL-DOS>   
              <CC>21</CC>   
              <YY>11</YY>   
              <MM>04</MM>   
              <DD>10</DD>   
              <CL-TAR-DRUG-CD>2</CL-TAR-DRUG-CD>   
              <CL-TAR-DRUG-DESC>desc</CL-TAR-DRUG-DESC>   
              <CL-NO>112153</CL-NO>   
              <CL-LINE>217615</CL-LINE>   
              <CL-AMT>31.58</CL-AMT>   
              <CL-TAR-AMT>0</CL-TAR-AMT>   
              <CL-PAID-PROV>31.58</CL-PAID-PROV>   
              <CL-PAID-MEM>0</CL-PAID-MEM>   
              <CL-RSN/>   
            </CLM>   
            <TOTALS>   
              <TOT-AMT>31.58</TOT-AMT>   
              <TOT-TAR-AMT>0</TOT-TAR-AMT>   
              <TOT-PAID-PROV>31.58</TOT-PAID-PROV>   
              <TOT-PAID-MEM>0</TOT-PAID-MEM>   
            </TOTALS>   
          </PROV>   
          <PROV>   
            <PR-NO>1</PR-NO>   
            <PR-NAME> pr</PR-NAME>   
            <REF-NO>0027579</REF-NO>   
            <CLM>   
              <REF>1</REF>   
              <CL-DEP-NO>04</CL-DEP-NO>   
              <CL-DEP-NAME>dep</CL-DEP-NAME>   
              <CL-DOS>09-JUN-11</CL-DOS>   
              <CC>21</CC>   
              <YY>11</YY>   
              <MM>06</MM>   
              <DD>09</DD>   
              <CL-TAR-DRUG-CD>99200 </CL-TAR-DRUG-CD>   
              <CL-TAR-DRUG-DESC>desc</CL-TAR-DRUG-DESC>   
              <CL-NO>1</CL-NO>   
              <CL-LINE>1</CL-LINE>   
              <CL-AMT>12</CL-AMT>   
              <CL-TAR-AMT>0</CL-TAR-AMT>   
              <CL-PAID-PROV>12</CL-PAID-PROV>   
              <CL-PAID-MEM>0</CL-PAID-MEM>   
              <CL-RSN/>   
            </CLM>   
            <TOTALS>   
              <TOT-AMT>12</TOT-AMT>   
              <TOT-TAR-AMT>0</TOT-TAR-AMT>   
              <TOT-PAID-PROV>12</TOT-PAID-PROV>   
              <TOT-PAID-MEM>0</TOT-PAID-MEM>   
            </TOTALS>   
          </PROV>   
        </ACC>   
        <RSN-LIST>   
          <RSN-ERR>????</RSN-ERR>   
          <RSN-DESC/>   
        </RSN-LIST>   
        <FIN-OTHER>   
          <FIN-DATE/>   
          <FIN-AMT>0</FIN-AMT>   
          <FIN-TT/>   
        </FIN-OTHER>   
        <GTOTALS>   
          <GTOT-AMT>125.78</GTOT-AMT>   
          <GTOT-TAR-AMT>67.2</GTOT-TAR-AMT>   
          <GTOT-PAID-PROV>125.78</GTOT-PAID-PROV>   
          <GTOT-PAID-MEM>0</GTOT-PAID-MEM>   
        </GTOTALS>   
        <MEM-GTOT>   
          <M-MESSAGE/>   
          <M-TOT-PAID-MEM>0</M-TOT-PAID-MEM>   
          <M-PMT-METH/>   
        </MEM-GTOT>   
        <FIN-BAL>   
          <FIN-BDATE>21-JUL-11</FIN-BDATE>   
          <FIN-BAMT>0</FIN-BAMT>   
          <FIN-BTT/>   
        </FIN-BAL>   
      </MEM-STMT>   
    </ROOT> 

I appreciate all the help.

THANK YOU

[g tsuji]

>in my xsl-fo file. I get a java.lang.OutOfMemoryError when I transform large data.
I take it meaning there is no OutOfMemoryError when you transform a smaller set of data? If that has not been checked yet, I would suppose the first thing to do is to test the templates with a smaller data set.

More particularly, if it is established already that it is the PROVIDER where the problem arise, you've to load a normal data set and a smaller data set with
ROOT/MEM-STMT/COUNTRY ='MU'
and
ROOT/MEM-STMT/FIN-BAL/FIN-BAMT !='0'
satisfied.

If smaller data set still provoke out of memory error, you have to check the named templated PROVIDER. It is easy to make out of memory error if, for instance, somewhere somehow there might be a recursive call... Hence, its source of trouble is more algorithm than data set.

If you are satisfied no recursion or algorithmic problem and that smaller data set for PROVIDER being called is free of memory error, you can then varying the strategy of how to prepare the Source for the transformation. You can have DOMSource which has the bigger memory footprint. If you're using that, change the algorithm to feed SAXSource to the transformer...

Those are what I would look into...

[Kamohelo Mofokeng]

Hi g tsuji,

Thanks for the response - much appreciated.

>>> I take it meaning there is no OutOfMemoryError when you transform a smaller set of data?

Yes, the transformation works fine for small data sets - the memory problem arises when dealing with large data sets.

>>> You can have DOMSource which has the bigger memory footprint. If you're using that, change the algorithm to feed SAXSource to the transformer...

I'm in the process of changing my code to make use of DOMSource, currently it looks as follows:

// the XML file from which we take the name
		final StreamSource source = new StreamSource(xml);
			
		// creation of transform source
		StreamSource transformSource = null;//new StreamSource(xlsIs);
		try {
			transformSource = (StreamSource)new CustomURIResolver().resolve(href, base);
		} catch (TransformerException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}		
		
                                // create an instance of fop factory
		FopFactory fopFactory = FopFactory.newInstance();
		
                                //a user agent is needed for transformation
		FOUserAgent foUserAgent = fopFactory.newFOUserAgent();
		
		// to store output
		ByteArrayOutputStream outStream = new ByteArrayOutputStream();
		
		TransformerFactory factory = TransformerFactory.newInstance();
		factory.setURIResolver(new CustomURIResolver());
		
		final Transformer xslfoTransformer;
		
		try {
				xslfoTransformer = factory.newTransformer(transformSource);
				
				System.out.println("Transformer xslfoTransformer = "  + xslfoTransformer);
				
				Fop fop = fopFactory.newFop
					(MimeConstants.MIME_PDF, foUserAgent, outStream);
				
				System.out.println("Fop fop = "  + fop);
				
				// Resulting SAX events (the generated FO) 
				// must be piped through to FOP
		        final Result res = new SAXResult(fop.getDefaultHandler());

		        System.out.println("Result res = "  + res);
		        
		        // Start XSLT transformation and FOP processing
		        System.out.println("Start XSLT transformation and FOP processing...");
				// everything will happen here..
			
		        messageLabel.setText("");
		        messageLabel.setText("PDF Transformation Started...");
		        
		       xslfoTransformer.transform(source, res);
		       
		       // if you want to get the PDF bytes, use the following code
			   //return outStream.toByteArray();
		       	
				System.out.println("Finished XSLT transformation and FOP processing");

				// if you want to save PDF file use the following code
				File pdffile = new File(outputPath+outputFileName);
				OutputStream out = new java.io.FileOutputStream(pdffile);
				
		        out = new java.io.BufferedOutputStream(out);
		        
		        FileOutputStream str = new FileOutputStream(pdffile);
		        str.write(outStream.toByteArray());
		        
		        out.flush();
		        str.close();
		        out.close();
		        bin.close(); 
			    output.close();
			    temp.delete();
		                        
		        System.out.println("Saved PDF file");            
		        messageLabel.setText("Saved PDF file!!");
		
		}catch(Exception e){
				e.printStackTrace();
		}

Any ideas how to go about it? I was thinking of:

final SAXSource sax = new SAXSource();
		
		SAXSource.sourceToInputSource(source);

THANKS

[g tsuji]

I could sketch a well stripped down layout of what should appear in coding. Nevertheless, I would prefer to dissect the process in tandem of first outputting the fo and then eventually processing the fo. I think you can handle the 2nd part adequately. The first part can look something like below. By this, one can have a good idea which part eats up the memory out-of-control.

XMLReader reader=XMLReaderFactory.createXMLReader();
Source source=new SAXSource(reader, new InputSource(xmlfile));  //your xml might need adaptation somehow or can be directly plug into xml here

OutputStream out=new ByteArrayOutputStream();
Result result=new StreamResult(out);

Source style = new StreamSource(xslfile);
TransformerFactory factory=TransformerFactory.newInstance();
Transformer xslfoTransformer=factory.newTransformer(style);    //your transformSource might be adapted to xslfile or style

xslfoTransformer(source, result);
//the result can then be directed to a JTextPane or else for inspection or redirected to a file...

If that remains within the capacity of memory available, you can start look into how fop is memory hungry.

[Kamohelo Mofokeng]

Thanks g tsuji, I really appreciate the help

I have changed my code as follows:

// the XML file from which we take the name   
        final Source source=new SAXSource(reader, new InputSource(xml+""));   
               
        // creation of transform source   
        StreamSource transformSource = null;//new StreamSource(xlsIs);   
        try {   
            transformSource = (StreamSource)new CustomURIResolver().resolve(href, base);   
        } catch (TransformerException e1) {   
            // TODO Auto-generated catch block   
            e1.printStackTrace();   
        }          
           
                                // create an instance of fop factory   
        FopFactory fopFactory = FopFactory.newInstance();   
           
                                //a user agent is needed for transformation   
        FOUserAgent foUserAgent = fopFactory.newFOUserAgent();   
           
        // to store output   
        ByteArrayOutputStream outStream = new ByteArrayOutputStream();   
           
        TransformerFactory factory = TransformerFactory.newInstance();   
        factory.setURIResolver(new CustomURIResolver());   
           
        final Transformer xslfoTransformer;   
           
        try {   
                xslfoTransformer = factory.newTransformer(transformSource);   
                   
                System.out.println("Transformer xslfoTransformer = "  + xslfoTransformer);   
                   
                Fop fop = fopFactory.newFop   
                    (MimeConstants.MIME_PDF, foUserAgent, outStream);   
                   
                System.out.println("Fop fop = "  + fop);   
                   
                // Resulting SAX events (the generated FO)    
                // must be piped through to FOP   
               final Result res = new StreamResult(outStream);   
  
                System.out.println("Result res = "  + res);   
                   
                // Start XSLT transformation and FOP processing   
                System.out.println("Start XSLT transformation and FOP processing...");   
                // everything will happen here..   
               
                messageLabel.setText("");   
                messageLabel.setText("PDF Transformation Started...");   
                   
               xslfoTransformer.transform(source, res);   
                  
               // if you want to get the PDF bytes, use the following code   
               //return outStream.toByteArray();   
                   
                System.out.println("Finished XSLT transformation and FOP processing");   
  
                // if you want to save PDF file use the following code   
                File pdffile = new File(outputPath+outputFileName);   
                OutputStream out = new java.io.FileOutputStream(pdffile);   
                   
                out = new java.io.BufferedOutputStream(out);   
                   
                FileOutputStream str = new FileOutputStream(pdffile);   
                str.write(outStream.toByteArray());   
                   
                out.flush();   
                str.close();   
                out.close();   
                bin.close();    
                output.close();   
                temp.delete();   
                                   
                System.out.println("Saved PDF file");               
                messageLabel.setText("Saved PDF file!!");   
           
        }catch(Exception e){   
                e.printStackTrace();   
        }  

3 scenarios play out when I run this code.

When I use:

final Result res = new SAXResult(fop.getDefaultHandler());

I run out of memory - pdf file is not created (for large data sets).

When I use:

final Result res = new SAXResult();

pdf file is not created (for large data sets) & I get the exception:

java.lang.IllegalArgumentException: handler can not be null for a SAXResult
	at org.apache.xalan.transformer.TransformerImpl.createSerializationHandler(TransformerImpl.java:1130)
	at org.apache.xalan.transformer.TransformerImpl.createSerializationHandler(TransformerImpl.java:1060)
	at org.apache.xalan.transformer.TransformerImpl.transform(TransformerImpl.java:1279)
	at org.apache.xalan.transformer.TransformerImpl.transform(TransformerImpl.java:1262)
	at com.momaf.tiermuis.applet.health.StatementGeneratorApplet.genereateClientPDF(Unknown Source)
	at com.momaf.tiermuis.applet.health.StatementGeneratorApplet.access$1200(Unknown Source)
	at com.momaf.tiermuis.applet.health.StatementGeneratorApplet$4.actionPerformed(Unknown Source)
	at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
	at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)
	at java.awt.Component.processMouseEvent(Unknown Source)
	at javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.awt.Component.processEvent(Unknown Source)
	at java.awt.Container.processEvent(Unknown Source)
	at java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.awt.EventQueue.access$000(Unknown Source)
	at java.awt.EventQueue$1.run(Unknown Source)
	at java.awt.EventQueue$1.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
	at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue$2.run(Unknown Source)
	at java.awt.EventQueue$2.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.AccessControlContext$1.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)

When I use:

final Result res = new StreamResult(outStream);

a pdf file is created but when I try to open it I get the error:

There was an error opening this document. The file is damaged and could not be repaired.

What am I missing here?

THANKS

[g tsuji]

I read the code with some difficulty following the logic, and I am not necessary agree with some of the constructions (the out to BufferedOutputStream(out)...). Some code progession has lost me squarely, and I might think it wrong. But since the code is not mine, I might be excused for not paying enough thoughts into it to understand. So I decide not to modify line after line with comments but only a wholesale modification below.

This is a version that I change and move around code lines. Watch carefully, including the ordering of lines to successively establish the objects. I do not invent any new variable names. They are all yours.

final Source source=new SAXSource(reader, new InputSource(xml+""));

StreamSource transformSource = null;  //new StreamSource(xlsIs);
try {
    transformSource = (StreamSource)new CustomURIResolver().resolve(href, base);
} catch (TransformerException e1) {
    e1.printStackTrace();
}           

FopFactory fopFactory = FopFactory.newInstance();
FOUserAgent foUserAgent = fopFactory.newFOUserAgent();
              
TransformerFactory factory = TransformerFactory.newInstance();      
factory.setURIResolver(new CustomURIResolver());      
              
final Transformer xslfoTransformer;      
              
try {      
    xslfoTransformer = factory.newTransformer(transformSource);      
	
    File pdffile = new File(outputPath+outputFileName);      
    OutputStream out = new BufferedOutputStream(new FileOutputStream(pdffile));
    Fop fop = fopFactory.newFop(MimeConstants.MIME_PDF, foUserAgent, out);      
                  
    final Result res=new SAXResult(fop.getDefaultHandler());

    messageLabel.setText("");      
    messageLabel.setText("PDF Transformation Started...");      
                      
    xslfoTransformer.transform(source, res);      

    out.flush();      
    out.close();      
                                      
    System.out.println("Saved PDF file");                  
    messageLabel.setText("Saved PDF file!!");      
              
}catch(Exception e){      
	e.printStackTrace();      
}    

[Kamohelo Mofokeng]

Hi g tsuji,

Please forgive my not-so-neat code - I've been chopping & changing while trying to get to the solution. Interestingly though, this is the code that's been working all along & still working for smaller data sets.

Anyway, I've made the changes according to the code you provided. I see that major difference in your arranged code is that you create both the file & bos objects before the transformation whereas I created them after the transformation.

With these changes in place I still get OutOfMemoryError, stack trace:

Exception in thread "AWT-EventQueue-2" java.lang.OutOfMemoryError: Java heap space
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
	at java.lang.Class.getMethod0(Unknown Source)
	at java.lang.Class.getMethod(Unknown Source)
	at org.apache.commons.logging.LogFactory.getContextClassLoader(LogFactory.java:438)
	at org.apache.commons.logging.LogFactory$1.run(LogFactory.java:222)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.commons.logging.LogFactory.getFactory(LogFactory.java:218)
	at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
	at org.apache.fop.layoutmgr.inline.BasicScaledBaselineTable.<init>(BasicScaledBaselineTable.java:37)
	at org.apache.fop.layoutmgr.inline.ScaledBaselineTableFactory.makeFontScaledBaselineTable(ScaledBaselineTableFactory.java:44)
	at org.apache.fop.layoutmgr.inline.ScaledBaselineTableFactory.makeFontScaledBaselineTable(ScaledBaselineTableFactory.java:56)
	at org.apache.fop.layoutmgr.inline.AlignmentContext.<init>(AlignmentContext.java:179)
	at org.apache.fop.layoutmgr.inline.LineLayoutManager.getNextKnuthElements(LineLayoutManager.java:580)
	at org.apache.fop.layoutmgr.BlockStackingLayoutManager.getNextKnuthElements(BlockStackingLayoutManager.java:294)
	at org.apache.fop.layoutmgr.BlockLayoutManager.getNextKnuthElements(BlockLayoutManager.java:116)
	at org.apache.fop.layoutmgr.table.TableCellLayoutManager.getNextKnuthElements(TableCellLayoutManager.java:170)
	at org.apache.fop.layoutmgr.table.RowGroupLayoutManager.createElementsForRowGroup(RowGroupLayoutManager.java:120)
	at org.apache.fop.layoutmgr.table.RowGroupLayoutManager.getNextKnuthElements(RowGroupLayoutManager.java:60)
	at org.apache.fop.layoutmgr.table.TableContentLayoutManager.getKnuthElementsForRowIterator(TableContentLayoutManager.java:228)
	at org.apache.fop.layoutmgr.table.TableContentLayoutManager.getNextKnuthElements(TableContentLayoutManager.java:138)
	at org.apache.fop.layoutmgr.table.TableLayoutManager.getNextKnuthElements(TableLayoutManager.java:247)
	at org.apache.fop.layoutmgr.BlockStackingLayoutManager.getNextKnuthElements(BlockStackingLayoutManager.java:294)
	at org.apache.fop.layoutmgr.BlockLayoutManager.getNextKnuthElements(BlockLayoutManager.java:116)
	at org.apache.fop.layoutmgr.BlockContainerLayoutManager.getNextKnuthElements(BlockContainerLayoutManager.java:290)
	at org.apache.fop.layoutmgr.BlockStackingLayoutManager.getNextKnuthElements(BlockStackingLayoutManager.java:294)
	at org.apache.fop.layoutmgr.BlockLayoutManager.getNextKnuthElements(BlockLayoutManager.java:116)
	at org.apache.fop.layoutmgr.FlowLayoutManager.getNextKnuthElements(FlowLayoutManager.java:107)
	at org.apache.fop.layoutmgr.PageBreaker.getNextKnuthElements(PageBreaker.java:145)
	at org.apache.fop.layoutmgr.AbstractBreaker.getNextBlockList(AbstractBreaker.java:552)
	at org.apache.fop.layoutmgr.PageBreaker.getNextBlockList(PageBreaker.java:137)
	at org.apache.fop.layoutmgr.AbstractBreaker.doLayout(AbstractBreaker.java:302)

I think the problem lies with the line:

final Result res = new SAXResult(fop.getDefaultHandler());

I don't why it is or how to move passed it.

THANKS

[g tsuji]

[0] It is unavoidable that the fop engine processing consumes heap space. There are some notes out there purporting that page-sequence will consume heap space without releasing... until it is switched to another. Hence, there is a urban wisdom saying something like that you have to try not use the same page-sequence master for a whole lot of pages. This is worth considering. Maybe you can try consistently use a fo:page-sequence for even page and another, amid the same in terms of fo:flow or else, for odd page etc... I say this taking into consideration that you've already done adding option -Xms and -Xmx up to Giga Bytes...

[1] I have tried to emulate the problem with 2K+ pages with single fo:page-sequence and fop can handle adding -Xms512m order of magnitude. But, the page is relatively simple without graphic and user generated fonts. But it is conceivable page size continuously increase until breaking all possible resource size of the machine... Hence, maybe the memory model of fop could be redesign one-day by those in that specialized field.

[2] If you are using fop-1.0 (it seems not available in fop-0.95), there is a ConserveMemoryPolicy setting for FOUserAgent, something like this:

foUserAgent.setConserveMemoryPolicy(true);

You can try if that serves any purpose. My testing on fop-1.0 has no luck on noticeable effect memory-wise in this regard.

[3] It is a certainty that however large resource fop can handle, it would be exhausted by arbitrarily large data set and fo document. Hence, I am not able to help beyond this physical limit.

[Kamohelo Mofokeng]

[0] It is unavoidable that the fop engine processing consumes heap space. There are some notes out there purporting that page-sequence will consume heap space without releasing... until it is switched to another. Hence, there is a urban wisdom saying something like that you have to try not use the same page-sequence master for a whole lot of pages. This is worth considering. Maybe you can try consistently use a fo:page-sequence for even page and another, amid the same in terms of fo:flow or else, for odd page etc... I say this taking into consideration that you've already done adding option -Xms and -Xmx up to Giga Bytes...

I honestly thought this was a better option after I investigated all the other alternatives. I was also following the example: http://www.scriptorium.com/whitepapers/xslfo/xslfo_4.html but I couldn't get it to work for my environment. At this point, I have no preference over which alternative I'd like to implement just as long it works.

[1] I have tried to emulate the problem with 2K+ pages with single fo:page-sequence and fop can handle adding -Xms512m order of magnitude. But, the page is relatively simple without graphic and user generated fonts. But it is conceivable page size continuously increase until breaking all possible resource size of the machine... Hence, maybe the memory model of fop could be redesign one-day by those in that specialized field.

If I can generate 1K+ pages that would be ideal. As it is, I'm struggling to generate 500 pages. It's mainly because of graphics/PNG images (max. size 231 KB) & user defined fonts. Is there a way that I can change fonts to make the transformation less resource intensive? Problem is I don't have control over the stylesheet format.

[2] If you are using fop-1.0 (it seems not available in fop-0.95), there is a ConserveMemoryPolicy setting for FOUserAgent, something like this:

I'm using fop-0.93 but I think this approach is worth checking out. I'll upgrade & test - hopefully this works out.

[3] It is a certainty that however large resource fop can handle, it would be exhausted by arbitrarily large data set and fo document. Hence, I am not able to help beyond this physical limit.

I'm hoping it won't come to this - I don't have other viable alternatives.