Young Choi

Greenhorn
+ Follow
since Aug 22, 2009
Merit badge: grant badges
For More
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
0
Received in last 30 days
0
Total given
0
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Young Choi

You can download KIX32.EXE and use the KIX script for using it on DOS command line.
It can display a pop-up screen for a command line based process run.
14 years ago
Thanks a lot Bill!
The way you suggested was exactly right.
I used DataInputStream to read the file into a byte[] with fixed (or caculated) length for each chunk, and it works beautiful.
So, reading a file of one huge record (or line) is not a problem any more in Java. Now a remaining tiny issue is about the time to write into an output file with 4.76GB / 215 bytes records. But no quick way there would be I suppose.

Do appreciate again!
14 years ago
Thank you for your attention!
Yes, the file's logical record has a fixed format, and each record size is 215 bytes.
In that case, can I read in 4.76GB file using Scanner class? I did not know about Scanner class until I saw your comment here by the way as I still use SDK1.4.
If so, could you post a sample Scanner usage for handling huge file of one big line like I have?

Thanks again.
14 years ago
BufferedReader is good for reading in a huge file of multiple records.
But if it is a huge file of "one record" (one line concatenated with multiple records) like 3GB to read in, even BufferedReader with -Xms1g -Xmx3g command line java execution option won't be able to process the data.
This is my situation that I have to process a 4.76GB file of one huge line (the file provider has sent us that kind of file concatenated into one line!).
Maybe I will have to split the file to smaller pieces for processing it, then I will have to merge them into one file to send it to the next system.

Is there any other good idea other than splitting it to pieces?
14 years ago