• Post Reply Bookmark Topic Watch Topic
  • New Topic

Problem with reading a huge text file efficiently  RSS feed

 
buntha Choudhary
Ranch Hand
Posts: 136
Android Eclipse IDE Windows XP
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have a huge text file and my job is to read the content of the file and add those records in a JTable.
Simple reading of such a huge file is very inefficient.
What is the most efficient possible way to solve this issue and please provide an example.
 
Christian Dillinger
Ranch Hand
Posts: 200
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What is "huge"? How many lines? How many columns? Do you have to show everything in your table or just some fields from each line?
 
buntha Choudhary
Ranch Hand
Posts: 136
Android Eclipse IDE Windows XP
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
consider it is about 50000 lines with having 3 columns.
I need to show all these records but can't use any general reading technique.
 
Stephan van Hulst
Saloon Keeper
Posts: 7993
143
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We're talking what 200 - 500 kb here?

Why can't you use a simple BufferedReader? Have you tried it? Doesn't it perform well enough?
 
Christian Dillinger
Ranch Hand
Posts: 200
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If it's 100 byte per line we are talking about 5 MB! But that's by far not to much hold in memory... I'd read to a List<String[]> and put that list into a simple TableModel.
 
Steve Luke
Bartender
Posts: 4181
22
IntelliJ IDE Java Python
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You might find using the NIO mapped files a means of speeding up the I/O functions.
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Wait a second

You are trying to present a swing JTable with 50,000 lines??

How in the name of sanity do you expect somebody to navigate in that?

Isn't there some kind of overall structure you can use to present smaller JTables? Maybe some sort of partial match or alphabetic selection?

Bill


 
J. Insi
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
i thought that what he did was,
50,000 lines as a total.

But in the application runs... it was divided into several pieces... 100 lines per page perhaps?
well, not sure.

But yes, using the buffer method works fine for reading at the time IO works...
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!