Win a copy of Kotlin in Action this week in the Kotlin forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Trouble reading large file  RSS feed

 
Johannes Thorén
Ranch Hand
Posts: 64
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm trying to put about half a million strings into a linkedlist.
Im reading using bufferedReader, but it just keeps loading and nothing happens, or at best i get a java heap memory error.

Here's the code i use:



Any idea how i could solve the problem? I need every line to be read since i have to choose a random word for the game.
 
Jeff Verdegan
Bartender
Posts: 6109
6
Android IntelliJ IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No, you don't actually need to read the whole thing into memory at once. Depending on your specific requirements, there are different approaches you could take. However, if you want to start with the simple approach of just reading it all in, then you have two options:

1) Increase the JVM's max available heap at startup time with the -Xmx option. For example, java -Xmx 512m -cp whatever Myclass

2) Start with a smaller file--much, much, much smaller--until you've verified that the logic of your program is correct. This is a good idea regardless of what approach you finally decide on, as it will make testing much simpler.
 
Johannes Thorén
Ranch Hand
Posts: 64
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jeff Verdegan wrote:No, you don't actually need to read the whole thing into memory at once. Depending on your specific requirements, there are different approaches you could take. However, if you want to start with the simple approach of just reading it all in, then you have two options:

1) Increase the JVM's max available heap at startup time with the -Xmx option. For example, java -Xmx 512m -cp whatever Myclass

2) Start with a smaller file--much, much, much smaller--until you've verified that the logic of your program is correct. This is a good idea regardless of what approach you finally decide on, as it will make testing much simpler.


The program works perfect if i use a list with lets say 20 words. It's a hangman game, but the wordlist have like 400k + rows. A part of my task is to encrypt and decrypt all words in the file (seperate program).

But then in the game i need to read in all words, choose a random word for the game, decrypt it and present it to the player.
 
Paul Clapham
Sheriff
Posts: 22480
43
Eclipse IDE Firefox Browser MySQL Database
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Johannes Thorén wrote:The program works perfect if i use a list with lets say 20 words.


I'm surprised. It looks to me like you're only putting half of the words into the list. But I suppose that testing wouldn't really expose that sort of error, since selecting random items from half of the words is almost indistinguishable from selecting random items from all of the words.
 
Johannes Thorén
Ranch Hand
Posts: 64
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Paul Clapham wrote:
Johannes Thorén wrote:The program works perfect if i use a list with lets say 20 words.


I'm surprised. It looks to me like you're only putting half of the words into the list. But I suppose that testing wouldn't really expose that sort of error, since selecting random items from half of the words is almost indistinguishable from selecting random items from all of the words.


Yeah i think its more of an iterationproblem. Changed the code to this:



Since i only need one word for the game, i thought of reading just one word from the file. (altho i'm going to have to deal with the problem of encrypting every word on the file later). But it still just keeps loading and loading.>
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!