I am facing an issue while delivering a file more than 2gb using the API java.nio.channels.FileChannel.transferFrom. The size of the file copied from source is only till 2gb.
Kindly let me know if anyone could help me out with this.
Basically I need to find out about some JAva api which could help in coping files more than 2 Gb.
This java.nio.channels.FileChannel.transferFrom is basically transferring only 2 gb of the data and hence there is mismatch between the source file and destination file size.
So I am assuming it does not alllow file copy for more tha 2 gb...So if any one has any idea and then please do let me know...
I am not aware of a limit on the amount of data that the Java IO API can handle. You are probably running into a file system limit. What's the file system on the destination?
Hi Joe Thanks for the reply.....
I am checking the file size limit with the system. So according to you the there cannot be any issue with the java api in delivering files over 2 gb size. are there any specific ways or any methods or any API by which we could deliver such huge files?
Please let me know.
how about the available memory in terms of total RAM and allocated size to JVM? will it be of some help if being fine tuned?
As far as I have experienced, the I/O related stuff behave differently on different platforms. One good example is the way /tmp file works. Have you checked the documentation on the target platform?
Are you calling transferFrom() in a loop, checking the return value to ensure that all bytes are transferred? The API does indicate it's possible that not all bytes are transferred in one call, though it's vague about why this may occur. I would do something like this:
There seems to be no size limit in the destination system.
the filesize limit that you mean witth 2 gb files is in FAT 16 Filesystem (like MS DOS 3.x)
the FAT32 Filesystem has a file limit with 4GB, (like WIN95,...)
we are on UNIX (Solaris) there are know filesize limits on UFS or Storenext Filesystems
Raghavan Muthu wrote:how about the available memory in terms of total RAM and allocated size to JVM? will it be of some help if being fine tuned?
If he were running into a RAM issue, I'd expect to see an OutOfMemory exception. Now there is a size limit on a MappedByteBuffer: Integer.MAX_VALUE (which just happens to be 2GB), but that shouldn't affect a plain transfer.
Sup, are you catching Exceptions correctly?
Have you tried using the plain java.io classes to accomplish this copy?
Have you tried a non-Java method to copy the file?
Do you have some code so we can reproduce this problem?
I tried the for loop but facing the same issue. Only 2gb of the file is getting copied on the destination. i tried this:
srcChannel = new FileInputStream(sourceFile).getChannel();
destChannel = new FileOutputStream(fileDeliveryTemp).getChannel();
for (long n = 0, s = srcChannel.size(); n < s; ) {
s_logger.debug("inside for loop " + destChannel.size());
n += destChannel.transferFrom(srcChannel, n, s - n);
}
Have you tried closing the stream and/or channel? Either should be sufficient, and I should have asked this earlier. That should be enough - but if it doesn't work, also try FileChannel's force() method. I don't think it matters much whether you use force(true) or force(false).
If it's still not working, what does your new debug statement tell you? Better yet, let's add a few more debugs:
What does the log output tell you? How many times is the loop really executing? What is the total number of bytes that the methods claims have been written (n)?
Post by:autobot
Rototillers convert rich soil into dirt. Please note that this tiny ad is not a rototiller:
a bit of art, as a gift, the permaculture playing cards