Win a copy of Kotlin in Action this week in the Kotlin forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

how many words ?  RSS feed

 
Shraddha Jain
Greenhorn
Posts: 27
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There is this HUGE csv file. i wanna know how many comma separated values it has. this file is huge ,many many giga bytes huge, and memory is dear, so reading all the values into a collection isnt an option at all..
any thoughts on this one?
Thanks.
 
David O'Meara
Rancher
Posts: 13459
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
cat|wc -w
ie if it can't be done in Java, find a technology that fits.

Or to paraphrase myself:
When your only tool is a hammer, all of your problems start to look like nails.
 
Shraddha Jain
Greenhorn
Posts: 27
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
To paraphrase myself, what if your only tool is the hammer?
 
Manuel Moons
Ranch Hand
Posts: 229
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You can read the file line by line and parse each line separately.

Just make up the totals in memory and do not save the encountered data.

Just use a simple FileReader (or FileInputStream). There is nothing complex about this.
 
Srinivas Kalvala
Ranch Hand
Posts: 257
Firefox Browser Hibernate Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,

Use FileReader and wrap with BufferedReader, the file swapping between primary memory and secondary memory will be taken care automatically, just go on reading the number of words in a loop by reading a line by line.

I have tested with some huge file, but not with GB ....
 
Ernest Friedman-Hill
author and iconoclast
Sheriff
Posts: 24217
38
Chrome Eclipse IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No idea why this is in JiG (Advanced.) Moving to Java in General (Beginner).
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!