• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Java Error while opening a large file of 2.2GB in appending mode, inside a zipfile used JAVA7 NIO

 
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
While I'm trying to open a csv file of 2.2GB in append mode which is in a zip file whose size is 144MB, I get the following exception.

java.lang.IllegalArgumentException: Negative initial size: -2030790440
at java.io.ByteArrayOutputStream.<init>(ByteArrayOutputStream.java:74)
at com.sun.nio.zipfs.ZipFileSystem.getOutputStream(ZipFileSystem.java:1371)
at com.sun.nio.zipfs.ZipFileSystem.newOutputStream(ZipFileSystem.java:516)
at com.sun.nio.zipfs.ZipPath.newOutputStream(ZipPath.java:792)
at com.sun.nio.zipfs.ZipFileSystemProvider.newOutputStream(ZipFileSystemProvider.java:285)
at java.nio.file.Files.newOutputStream(Files.java:170)
at java.nio.file.Files.newBufferedWriter(Files.java:2705)
at

Here is the code snippet I'm using :


note :
file already exists
It seems like BufferWriter's int max limit is reached while reading the size of the file hence the exception.
 
Rancher
Posts: 4801
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Looking at the code (or at least a version of it) for ZipFileSystem, it creates the ByteArrayOutputStream based on the size of the Entry, which is a long and is defined as:
"uncompressed size of entry data"
which would be your 2.2Gb, presumably.

This value is cast to an int, since that's what the constructor for the stream expects (which strikes me as slightly dangerous)...and in your case goes boom.
So that's the cause. It's the ByteArrayOutputStream which has the limit, not BufferedWriter.
 
Bartender
Posts: 3323
86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Dave Tolls wrote:This value is cast to an int, since that's what the constructor for the stream expects (which strikes me as slightly dangerous)
So that's the cause. It's the ByteArrayOutputStream which has the limit, not BufferedWriter


To add to this explanation: The maximum array that can be defined in Java is Integer.MAX_VALUE and as ByteArrayOutputStream creates a byte array to store the data in you are unfortunately limited to that size.
 
Dave Tolls
Rancher
Posts: 4801
50
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tony Docherty wrote:

Dave Tolls wrote:This value is cast to an int, since that's what the constructor for the stream expects (which strikes me as slightly dangerous)
So that's the cause. It's the ByteArrayOutputStream which has the limit, not BufferedWriter


To add to this explanation: The maximum array that can be defined in Java is Integer.MAX_VALUE and as ByteArrayOutputStream creates a byte array to store the data in you are unfortunately limited to that size.



Ah, of course.
That fills in that bit.
Still, you think they'd check before doing the conversion and throw some exception or other that would be more meaningful.
 
Tony Docherty
Bartender
Posts: 3323
86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Dave Tolls wrote:Still, you think they'd check before doing the conversion and throw some exception or other that would be more meaningful.


Yes, it does appear to be something of an oversight.
 
And will you succeed? Yes you will indeed! (98 and 3/4 % guaranteed) - Seuss. tiny ad:
a bit of art, as a gift, that will fit in a stocking
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic