• Post Reply Bookmark Topic Watch Topic
  • New Topic

searching a large text file >1gb  RSS feed

 
sree nidhi
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,

Can anyone suggest one what can be a efficient method to search a large text file >1gb .I tried using file channel but it is throwing an exception.Is there any other way??

Thank You
 
Joseph Macer
Ranch Hand
Posts: 63
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You'll need to be a bit more specific. A simple way to search a text file of arbitrary size is to simply step through it looking for a keyword. If you're getting an exception due to being out of memory, you have a memory leak somewhere, or are trying to read through the whole file at once.

For any operation, you can typically be efficient with time, memory, or I/O - usually you get to pick two of those three. If you don't have enough memory to do it quickly, you will take longer to do it a different way.
 
Omar Al Kababji
Ranch Hand
Posts: 357
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Since the file is very big, I would suggest to use a Scanner since it can do its scanning using an input stream and there will be no need to load all the file data to memory to make the search.

Scanner Documentation
 
Ulf Dittmer
Rancher
Posts: 42970
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What exception are you getting?

Also, what would the result of the search be? A line number? The line in which the search term occurs? Something else?

Using a library like Lucene may also help.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!