Win a copy of Transfer Learning for Natural Language Processing (MEAP) this week in the Artificial Intelligence and Machine Learning forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • Paul Clapham
  • Devaka Cooray
  • Bear Bibeault
Sheriffs:
  • Junilu Lacar
  • Knute Snortum
  • Liutauras Vilda
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Piet Souris
Bartenders:
  • salvin francis
  • Carey Brown
  • Frits Walraven

Syntax tree memory issues

 
Rancher
Posts: 157
14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm writing something of an IDE for a little-known language and I use a syntax tree to parse the code into logical representations. Essentially breaking it down into a variables section, functions, statement, and then finally argument to a statement. This approach works for most of the code files I've tested it against, but unfortunately some of my files are extremely large. As in, 100,000+ lines long. When it attempts to read these, it can run out of memory - especially because I made the tree immutable and so any code transformation needs to copy the tree (looking back on it, maybe a poor design decision).

Has anyone run into Java memory issues later? Are there best practices to increase efficiency or increase memory?
When I run my jar, I add the following command to it: -Xmx1g
Since I read that allows more memory use.
 
Marshal
Posts: 25452
65
Eclipse IDE Firefox Browser MySQL Database
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yeah, that tells the JVM to use 1 gigabyte for its heap. But if you're using a 64-bit version of Java (not too hard to arrange if you aren't) then you can use a much larger heap size. I just tried running a little test program with a 32-gigabyte heap, with no complaint from the JVM. Even larger heaps might be possible, I haven't tried.
 
Paul Clapham
Marshal
Posts: 25452
65
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Although... you can certainly tell the JVM that you want a 32-GB heap, but if you don't actually have 32 gigabytes of memory it will happily continue with whatever memory you actually have available.
 
    Bookmark Topic Watch Topic
  • New Topic