Writing large files to the servletoutputstream in chunks

[az ziz]

dear all,

If i have to download a large file, say > 200 mb, from the servlet if i use fileinputsream to read and write to the servletoutputstream i get outofmemory error. how do i split this file into various chunks and write them to the stream?
does using
response.setContentType("application/octet-stream");
solve the problem?
although i tried
outputStream = new GZIPOutputStream(response.getOutputStream(), 10240);
it didnt solve the problem.

thanks
V

[az ziz]

can this be solved using the multi-part response? how about the browser compatibility? i read somewhere long back that IE do not understand this response ?

V

[David Newton]

Without knowing how you're copying from one stream to the other it's impossible to help.

[az ziz]

i have few pdf files in the file server. This pdf may vary in sizes. i need to let the client download them though the servlet. i receive the file name from the request and read the file using FileInputStream. i write them to the ServletOutputStream.
during this operation i receive OutOfMemoryError.

kindly reply for any other clarification.

V

[David Newton]

When someone on a Java forum asks about "how you're doing something" it's almost *always* a request for code, or at the *very* least, pseudo-code. Copying a file should use essentially zero memory unless you're doing it wrong. Read into a buffer, write the buffer out, repeat until done. No memory other than the buffer is being used.

Personally, I almost always just use something from commons-io, but it does the exact same thing.

So again, without knowing what you're doing, it's impossible to know what you might be doing wrong.

[az ziz]

for your reference, please find the below codes that does the said work.

unnecessary codes are removed

pdfPrintList is the List containing the FileInputStream of the files to be sent to the client.

 


OutputStream outputStream = response.getOutputStream();
				response.setContentType("application/octet-stream");
				response.setHeader("Content-disposition",
						"attachment;filename=simplificationpolicy.pdf");
				response.setHeader("Content-Encoding", "gzip");
				// response.setContentLength(-1);
				outputStream = new GZIPOutputStream(outputStream, 10240);
				_joinPDFs(pdfPrintList, outputStream, false);
				try {
					outputStream.flush();
				} finally {
					if (outputStream != null) {
						try {
							outputStream.close();
						} catch (IOException ex) {
							ex.printStackTrace();
						}
					}
				}
				return;

 

below code will join the list of pdfs, this will use the itext lib.

protected void _joinPDFs(List<InputStream> pdfStreams,
			OutputStream outputStream, boolean paginate) {

		Document document = new Document();
		try {
			// Create a writer for the outputstream
			PdfWriter writer = PdfWriter.getInstance(document, outputStream);
			writer.setFullCompression();

			document.open();
			BaseFont bf = BaseFont.createFont(BaseFont.HELVETICA,
					BaseFont.CP1252, BaseFont.NOT_EMBEDDED);
			PdfContentByte cb = writer.getDirectContent(); // Holds the PDF
			// data

			PdfImportedPage page;
			int currentPageNumber = 0;
			int pageOfCurrentReaderPDF = 0;
			for (int i = 0; i < pdfStreams.size(); i++) {
				PdfReader pdfReader = new PdfReader(pdfStreams.get(i));

				// Create a new page in the target for each source page.
				while (pageOfCurrentReaderPDF < pdfReader.getNumberOfPages()) {
					pageOfCurrentReaderPDF++;
					document.setPageSize(pdfReader
							.getPageSizeWithRotation(pageOfCurrentReaderPDF));
					document.newPage();
					// pageOfCurrentReaderPDF++;
					currentPageNumber++;
					page = writer.getImportedPage(pdfReader,
							pageOfCurrentReaderPDF);
					cb.addTemplate(page, 0, 0);

				}
				pageOfCurrentReaderPDF = 0;
				System.out.println("now the size is: "
						+ pdfReader.getFileLength());
				outputStream.flush();
			}
			document.close();
		} catch (Exception e) {
			e.printStackTrace();
		} finally {
			if (document.isOpen())
				document.close();
		}
	}

thanks
V

[William Brogden]

Looks to me like it is the PDF combining that uses all the memory, not the servlet output stream.

Didn't you already write about this problem in the performance forum?

HERE!

WHY in the name of Alan Turing are you coming back with the same problem, after NOT taking the advice in the performance forum.

Bill