• Post Reply Bookmark Topic Watch Topic
  • New Topic

Multiple Readers/Streams on a single file  RSS feed

 
Rodney Woodruff
Ranch Hand
Posts: 80
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,
Is it possible to have multiple readers read the same file at the same time? I have a text file with over 47 million line of text.
I want to start up several readers that would each start at a different point in a file and then read to a certain point.
Does anyone have any suggestions on how I could do this?
-- Rodney
 
Ron Newman
Ranch Hand
Posts: 1056
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
As long as they are all reading, and nobody is writing, this should work.
 
Rodney Woodruff
Ranch Hand
Posts: 80
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ron,
Can you tell me how I could have reader 1 start at line 1000 and reader two start at line 3000?
Thanks alot.
-- Rodney
 
Ron Newman
Ranch Hand
Posts: 1056
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You'll want to create two FileReaders, wrap each one in a BufferedReader, and use readLine() to skip over the appropriate number of lines in each one.
 
Ron Newman
Ranch Hand
Posts: 1056
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Depending on what you're doing, you might find RandomAccessFile more to your liking.
 
Rodney Woodruff
Ranch Hand
Posts: 80
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I thought about using a RandomAccessFile but I have a problem going to a specific record in the file. For example, how would I use a RandomAccessFile to go to line 10,000 in a file?
This could be easy if every record had the same number of bytes. Unfortunately, they don't.
Any and all help is appreciated.
Thanks alot.
-- Rodney
[ September 23, 2002: Message edited by: Rodney Woodruff ]
 
Ron Newman
Ranch Hand
Posts: 1056
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You'd have to do the same thing -- call readLine() repeatedly in a loop.
However, let's say that you want to create three RandomAccessFile objects -- the first starting at line 3000, the second at line 7000, the third at line 10000.
Once you've created the first RandomAccessFile and located the beginning of line 3000, you can now call getFilePointer() to determine the byte offset of this position. When you create the second RandomAccessFile, you can call seek() to "fast-forward" to this same position. Now you only need to call readLine() 4000 times instead of 7000.
Similarly, you can call getFilePointer() on this second RandomAccessFile object in order to seek to that position with the third one.
There ought to be some way to do this with FileReader, but I don't see one. It seems like a deficiency in the Java I/O library.
 
David Weitzman
Ranch Hand
Posts: 1365
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
To skip a fixed number of bytes, you can use Reader.skip(long). For such a massive file, it's worth the trouble to write you're own implementation of FilterReader that allows you to skip lines without the overhead of creating String objects.
 
Ron Newman
Ranch Hand
Posts: 1056
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
But I don't see an equivalent to getFilePointer() for Readers.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!