Win a copy of Murach's Python Programming this week in the Jython/Python forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

File Channel  RSS feed

 
glen croteau
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,

i have been testing. I am after some seasoned advice please. I need to copy large quantities of jpg images. Stability is more important than performance but performance is also important.

I have been trying File Channels which are great.

Is the overhead of creating a Map too great for files only 4mb. My tests are about the same.

FileChannel in = new FileInputStream(srcFile).getChannel();
FileChannel out = new FileOutputStream(destFile).getChannel();
long size = in.size();
MappedByteBuffer buf = in.map(FileChannel.MapMode.READ_ONLY, 0, size);
out.write(buf);
in.close();
out.close();


Is it a stable practice to copy an entire file of 4 mb in one go or is it better to create smaller chunks and loop on the file. My tests indicate copying in one go is most efficient, but is it always reliable.


When I use a channel as follows are the FileInputStream and the FileOutputStream closed automatically. I read somewhere that it is not, however the docs are not clear and I have seen plenty of code like this on the web.

fcin = new FileInputStream(inFile).getChannel();
fcout = new FileOutputStream(outFile).getChannel();
fcin.transferTo(0, (int) fcin.size(), fcout);
fcin.close();
fcout.close();

Finally is it sensible to put the copy code in a loop and close the channels after the loop or must the channel be closed after I am done with each file?


....
while(true){
inFile=..
outFile=..
fcin = new FileInputStream(inFile).getChannel();
fcout = new FileOutputStream(outFile).getChannel();
fcin.transferTo(0, (int) fcin.size(), fcout);
}
fcin.close();
fcout.close();

Thanks
 
Nitesh Kant
Bartender
Posts: 1638
IntelliJ IDE Java MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,
We have an official policy on what makes an acceptable display name. I think your display name stands to violate that policy.

Coming to your questions.

sub lime:Is it a stable practice to copy an entire file of 4 mb in one go or is it better to create smaller chunks and loop on the file. My tests indicate copying in one go is most efficient, but is it always reliable.


Indeed that is true. You will be better off if you use transferTo() methods on the FileChannel because this will do all the transfer directly using the OS routines and not getting anything in the JVM at all. This will be more effective, performant and obviously bug free as you do not write any code

sub lime:When I use a channel as follows are the FileInputStream and the FileOutputStream closed automatically.

In fact it is the opposite, if you close the stream the corresponding channel is closed.

sub lime:Finally is it sensible to put the copy code in a loop and close the channels after the loop or must the channel be closed after I am done with each file?

I am not sure what you mean here. In the loop you will be assigning a new stream and a new channel for every iteration. You only close the channel when you come out and that too only the channel that you open in the last iteration. This will be creating havoc if this code is run many a times and with big iterations. Very soon you will run out of File descriptors as you do not close the streams.

Lastly, I am not too sure about the first approach you have shown for copying the file. I dont think MappedByteBuffer will be a good idea here. The best bet is to use transferTo methods on the file channel.
[ April 11, 2008: Message edited by: Nitesh Kant ]
 
glen croteau
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you very much for your reply. You have clarified plenty.

Still unclear. Given the following example is it ok to close the FileChannel only once in the finally block. If so does a call to fcin.close(); close the new FileInputStream(srcFile)?

Is it a better practice to create the FileInputStream / FileOuputStream objects and close them explicitly.

for example:

FileInputStream fis=null;
FileInputStream fos=null;

and in the finally block

if(fis)fis.close());
fcin.close();
if(fos)fos.close());
fcout.close();




 
Jim Yingst
Wanderer
Sheriff
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
[Glen]: When I use a channel as follows are the FileInputStream and the FileOutputStream closed automatically.

[Nitesh]: In fact it is the opposite, if you close the stream the corresponding channel is closed.


It turns out that closing either one also closes the other. However this is not clear from the documentation. The API for FileInputStream, FileOutputStream and RandomAccessFile tells us clearly that closing those objects closes the associated FileChannel. But the reverse does not appear to be stated anywhere in the API - even though it does seem to be true. So it's probably best to close the stream rather than the channel, to be safe. In practice though, I often do something like this:

I don't bother saving a reference to the input stream because I know that it will be closed too. This may be viewed as cheating, sort of, since it's relying on undocumented information. But it hasn't created any problems for me so far. I prefer not to clutter my code with variables I don't really need. Ultimately I think it's OK to close either the stream or the channel; whichever you prefer.
 
glen croteau
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you.
 
Nitesh Kant
Bartender
Posts: 1638
IntelliJ IDE Java MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
glen: Given the following example is it ok to close the FileChannel only once in the finally block.


In the above code, why are you instantiating a new input/output stream on every iteration? (assuming the order object remains the same.)
What condition terminates the loop? (I could only see an exception terminating it)
Answering your question. Yes, it is ok to close the file stream/channel once in finally if you instantiate the channel once. In a nutshell, you must close all the channels that you open.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!