Win a copy of The Little Book of Impediments (e-book only) this week in the Agile and Other Processes forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

OutOfMemoryError when accessing Zip File

 
Garret Schweitzer
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello all,

I get an OutOfMemoryError when accessing large ZipEntry(ies) in a ZIP file (using java.util.zip). It happens when trying to get the input stream. I can go through small entries and grab the input streams, and inflate the files inside, but if I hit a large file, it blows up.

Example:
... code before this....

if (!entry.isDirectory())
{
try
{
**************** blows up on this next line ******************
InputStream tempInputStream = zipFile.getInputStream(entry);
BufferedInputStream is = new BufferedInputStream(tempInputStream);

int currentByte;
// establish buffer for writing file
byte data[] = new byte[BUFFER];

// write the current file to disk
FileOutputStream fos = new FileOutputStream(destFile);
BufferedOutputStream dest = new BufferedOutputStream(fos, BUFFER);

// read and write until last byte is encountered
long totalBytes = 0;
long lastPrinted = 0;
Date startTime = new Date();
while ((currentByte = is.read(data, 0, BUFFER)) != -1)
{
totalBytes += BUFFER;
if ((totalBytes) > (lastPrinted + (ONE_HUNDRED_K * 3)))
{
System.out.println("extracted... " + totalBytes);
lastPrinted = totalBytes;
}
dest.write(data, 0, currentByte);
}
dest.flush();
dest.close();
is.close();
... more code follows this ...


I'm guessing that the getInputStream() must read the entire file into memory somehow, but I'm not a Java Guru. I don't know how I can unzip large files since the getInputStream() method doesn't take buffer sizes as arguments.

Any ideas would be greatly appreciated.

Thanks,
Garret
 
Ramender Mall
Ranch Hand
Posts: 311
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,


InputStream tempInputStream = zipFile.getInputStream(entry);


My theory is, that getInputStream() will return a "stream", not data, so there shud not be a question of buffer at least here..as a stream consumes or delivers data...which YOU do using read() and write()...

hmmm,
sort of a no use reply...but HOPE it helps ..

Ramy....
[ December 28, 2005: Message edited by: Ramender Mall ]
 
Garret Schweitzer
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Just to clarify that post (which I did late last night), here's a more thoughtful version....







I have a Java issue and I don't know how to solve it, so I thought I'd run it by the folks here and see if anyone here knows what to make of it.

First: The (simplified) code below works fine on a PC, but not in a limited memory environment, like a PDA.

Second: I have no control over the Java environment, so I can't control startup switches. In fact, I'll be running in a WebSphere environment. This code will be running in a J2ME environment as well as a J2SE and J2EE environment, so it needs to be friendly.

The problem: It runs out of memory when inflating a large file (greater than ten megs) from a ZIP. If I have 14 files in the ZIP and two are larger than ten megs, it will inflate all but the two large ones.

Where it does NOT occur: Strangely enough, it doesn't happen at a controllable place, like a buffered read. It isn't happening at any place where I can control the number of bytes read.

Where it DOES occur: It's happening at the line where I try to get the input stream getInputStream() for an entry in the ZIP file. Specifically, the large (greater than ten megs) entry in the file.

Why it happens: I have some theories. 1) It may be standard practice to read the entire file during a getInputStream() method call. 2) In order for the inflator to determine strategy, it may have to read the entire file. 3) The blocksize for the stream is determined in some way by the size of the file.

What you can do to help me: If you know the answer, great, let me know. I need to finish this project. ALSO, tell me how I can find this stuff out on my own. If there is some method of seeing the source code behind getInputStream() in java.util.zip.ZipFile, that might lend some clues. Perhaps looking at the bytecode would be handy...if I knew how to intrepret it. If there is a workaround that is more memory-friendly, that would be outstanding. I'm easy.

I'm going to post code at the end of this message.

Thanks in advance for help.

Garret




 
Ramender Mall
Ranch Hand
Posts: 311
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
well,
matter seems to be way out of my sight...

But for source code, try this post...
it might help......
hopefully, atleast a bit more than the last reply...

Ramy...
[ December 28, 2005: Message edited by: Ramender Mall ]
 
Alexis Krenvalks
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
For work with zip files I usually use-file recovery zip,because this tool is free as far as I know and helped me many times.Moreover program can too extracts data and saves it to the hard disk while saving folders and files.

 
Tim Holloway
Saloon Keeper
Posts: 18367
56
Android Eclipse IDE Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Alexis,

Thanks for trying to be helpful, but http://faq.javaranch.com/java/DontWakeTheZombies
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic