• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

FileInputStream large files

 
Ranch Hand
Posts: 115
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Im developing in java 1.4.2. Im using a FileInputStream to read in (1MB) chunks of a large file and transfer them to another machine which is read to write the chunk to a file. Im getting an out of memory error reading in the file (after about 60mb) any ideas? Have a feeling that there is some zombie references sitting about inside of my reading loop.
Also are there any good mechanisms for checking memory usage?

Cheers

Tom
 
author and iconoclast
Posts: 24207
46
Mac OS X Eclipse IDE Chrome
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
A proper read loop should have constant memory use, so yes, there's probably a leak.

Traditionally, commercial tools like JProbe or OptimizeIt have been the best way to monitor and examine memory use; I understand there are some useful monitoring tools that come with Java 5, however.
 
Tom Hill
Ranch Hand
Posts: 115
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The out of memory error was being caused by an ObjectInputStream.
The chunks Im reading are being put into a Transfer object that contains some other useful information. I was using an ObjectIn/OutputStream to write the transfer objects between sever and client. I have discovered that references are not being updated. Calling reset causes various Exceptions. Is there an easier mechanism for transfering objects that may change during the reading file loop?
 
Tom Hill
Ranch Hand
Posts: 115
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
...because Im calling reset on the wrong stream at the wrong time!

Cheers guys, once again youve given me lots to think about!



Tom
 
Ranch Hand
Posts: 3061
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you still need help, please post some code to show us what you are doing. Paraphrasing often loses important details that we need to help you.

Layne
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic