Win a copy of Terraform in Action this week in the Cloud forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • Rob Spoor
  • Bear Bibeault
Saloon Keepers:
  • Jesse Silverman
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • Al Hobbs
  • salvin francis

Problem with reading a huge text file efficiently

 
Ranch Hand
Posts: 136
Android Eclipse IDE Windows XP
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have a huge text file and my job is to read the content of the file and add those records in a JTable.
Simple reading of such a huge file is very inefficient.
What is the most efficient possible way to solve this issue and please provide an example.
 
Ranch Hand
Posts: 200
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What is "huge"? How many lines? How many columns? Do you have to show everything in your table or just some fields from each line?
 
buntha Choudhary
Ranch Hand
Posts: 136
Android Eclipse IDE Windows XP
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
consider it is about 50000 lines with having 3 columns.
I need to show all these records but can't use any general reading technique.
 
Saloon Keeper
Posts: 13424
300
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
We're talking what 200 - 500 kb here?

Why can't you use a simple BufferedReader? Have you tried it? Doesn't it perform well enough?
 
Christian Dillinger
Ranch Hand
Posts: 200
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If it's 100 byte per line we are talking about 5 MB! But that's by far not to much hold in memory... I'd read to a List<String[]> and put that list into a simple TableModel.
 
Bartender
Posts: 4179
22
IntelliJ IDE Python Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You might find using the NIO mapped files a means of speeding up the I/O functions.
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Wait a second

You are trying to present a swing JTable with 50,000 lines??

How in the name of sanity do you expect somebody to navigate in that?

Isn't there some kind of overall structure you can use to present smaller JTables? Maybe some sort of partial match or alphabetic selection?

Bill


 
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
i thought that what he did was,
50,000 lines as a total.

But in the application runs... it was divided into several pieces... 100 lines per page perhaps?
well, not sure.

But yes, using the buffer method works fine for reading at the time IO works...
 
reply
    Bookmark Topic Watch Topic
  • New Topic