• Post Reply Bookmark Topic Watch Topic
  • New Topic

How to process big file?

 
Howie Jiang
Greenhorn
Posts: 26
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I need to process very big file..
If I use FileInputStream, how big can the file be?
If I want to split the file into smaller ones, how to do it?
If the file is at remote location, how can I get and process it using streaming technology?

Thanks very much!
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
FileInputStream will read one byte at a time or a chunk at a time from now until the cows come home, so it can handle any size file. If you're storing those bytes in memory you can surely run out, though, so your next question is right on the money.

To split a big file in pieces, you could use a loop like:

As far as remote access, how you can get it all depends on what the remote host supports. If you can mount it as a drive you can read it like any other file. If a human can open it with a browser your program could open it with a URL or HttpURLConnection. You can also fetch files with FTP and other protocols, but I've never even looked at the doc for that. You could even write your own socket-based file server on the remote end.

Any of that help?
[ December 27, 2004: Message edited by: Stan James ]
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!