• Post Reply Bookmark Topic Watch Topic
  • New Topic

Best Approch for copying small or large files in java  RSS feed

 
Jude Niroshan
Ranch Hand
Posts: 132
5
Eclipse IDE Java Postgres Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,
I have a situation where i want to copy some files. they can be small or large in size. I am new to java. So, i thought just to pass those files through byte array. Is there another alternative way to do this while considering the performance side? I am doing this in a desktop application.
 
Jesper de Jong
Java Cowboy
Sheriff
Posts: 16057
88
Android IntelliJ IDE Java Scala Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you're using Java 7 or newer you could use one of the copy(...) methods in class java.nio.file.Files; see the API documentation.
 
Jude Niroshan
Ranch Hand
Posts: 132
5
Eclipse IDE Java Postgres Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I thought most of the people have still using java 1.5. So,if i used methods in later versions, that would make some runtime errors. What might be the best solution in that case? Think like coping file, sizes in GigaBytes(3GB). If it is a good idea if i just put it into a byte array?
 
Campbell Ritchie
Marshal
Posts: 56529
172
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jude Niroshan wrote:I thought most of the people have still using java 1.5. . . .
There are many companies which intentionally stick to very old and tested versions of software. There are however recently discovered security vulnerabilities meaning such old versions are no longer safe to use in applications with access to the net. So update to JDK7u71/72 or 8u25.
 
Maneesh Godbole
Bartender
Posts: 11445
18
Android Eclipse IDE Google Web Toolkit Java Mac Ubuntu
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My suggestion: Do not reinvent the wheel.
Check out FileUtils#copyFile from apache commons whose version 2.2 runs on 1.5
 
Jesper de Jong
Java Cowboy
Sheriff
Posts: 16057
88
Android IntelliJ IDE Java Scala Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jude Niroshan wrote:Think like coping file, sizes in GigaBytes(3GB). If it is a good idea if i just put it into a byte array?

Trying to load a 3 GB file into a huge byte array all at once is certainly not a good idea, this will cause an OutOfMemoryError (unless you have lots of RAM, a 64-bit JVM and set the JVM memory settings appropriately).
 
Dave Tolls
Ranch Foreman
Posts: 3056
37
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Prior to the nio stuff then, you're looking at "read a load of bytes, write a load of bytes".
The size of the byte array used will depend entirely on the environment you are operating in.
For example the Buffered Streams/Readers/Writers use (I think) 8k as a default.
You might find larger arrays more efficient, but that will be down to you testing it out.
 
Jude Niroshan
Ranch Hand
Posts: 132
5
Eclipse IDE Java Postgres Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So, it seems like most of you guys advice me to stick to java 1.7. But i started to write my application in java 1.5. I am much more familiar to stick with java 5. So, in that case, don't i have a choice? Putting into a byte array seems like definitely not going to work. Please my teachers, give me a solution. I want to copy huge files from one location to another in java. can i split large files and built them up in my destination location?
 
Jesper de Jong
Java Cowboy
Sheriff
Posts: 16057
88
Android IntelliJ IDE Java Scala Spring
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jude Niroshan wrote:But i started to write my application in java 1.5. I am much more familiar to stick with java 5. So, in that case, don't i have a choice?

Java 7 is not totally different from Java 5, everything you know about Java 5 still works on Java 7. Besides that, Java 5 is already very old and not even supported anymore by Oracle, so why would you want to write a new application using such an old Java version, if you have the choice?

You could use the Apache Commons library which Maneesh mentioned. Another option is the Google Guava library, a very good library which contains many useful utility methods, including a file copy method.

If you want to implement it yourself, write something like this: allocate a buffer of a few KB (for example 8192 bytes - 8 KB) then in a loop read into the buffer from the input file, write the buffer to the output file, until you reach the end of the input file. So you copy the file block by block instead of trying to read the whole file into memory at once.

Note that the Apache Commons and Google Guava libraries are both open source, you can get the source code and look how it is done in those libraries. That would be a good opportunity to learn from someone else's source code.
 
Campbell Ritchie
Marshal
Posts: 56529
172
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The one possible reason for developing Java7 code on a Java5 JDK is to ensure backward compatibility. Agree: it isn't a good explanation.
 
Winston Gutkowski
Bartender
Posts: 10575
66
Eclipse IDE Hibernate Ubuntu
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jude Niroshan wrote:So, it seems like most of you guys advice me to stick to java 1.7. But i started to write my application in java 1.5. I am much more familiar to stick with java 5. So, in that case, don't i have a choice? Putting into a byte array seems like definitely not going to work. Please my teachers, give me a solution. I want to copy huge files from one location to another in java. can i split large files and built them up in my destination location?

Yes; and there are a raft of other solutions too - including Maneesh's (which is probably what I'd go with if you're not ready to upgrade).

There's one thing I'd say here, though: copying files is generally a system issue, since it's usually the system that defines what a "file" is and how it's stored on disk (or a network). Java gives you abstractions to those things but, since it's goal is to be platform-independent, don't expect it to be able to do all the things you might expect to be able to do with a targeted system command like 'cp' in Unix.

Just one possibility, if you want to roll your own, is to use a Pipe, to which you attach the Streams that define your input and output. That will take care of any buffering that's required without you having to write any complicated code to "chunk" your data; and may give you better throughput than simple "read a byte, write a byte" logic - but to be honest, I don't know because I've never tried it.

HIH

Winston
 
Jude Niroshan
Ranch Hand
Posts: 132
5
Eclipse IDE Java Postgres Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
THANKS EVERYONE!!! I AM IMPRESSED, ALL ADVICES FROM PEOPLE WHO HAVE AT LEAST A 1 COW

So, pipe seems like to be the solution. I have decided to move to java 1.7. Is there any kind of effect if those files may be any kind of format? i mean like media files(mp3,mp4, bluray,etc) or other document files, system files(.dill, .bat,etc) ? can i transfer anything through a pipe?
 
Paul Clapham
Sheriff
Posts: 22823
43
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You're just sending bytes, and it doesn't matter what people think those bytes mean. In other words, when you're just transferring files then paying attention to the meaning of their contents is the wrong thing to do. Just transfer all of the bytes, nothing more.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!