Win a copy of Programmer's Guide to Java SE 8 Oracle Certified Associate (OCA) this week in the OCAJP forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

FileInputStream large files

 
Tom Hill
Ranch Hand
Posts: 115
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Im developing in java 1.4.2. Im using a FileInputStream to read in (1MB) chunks of a large file and transfer them to another machine which is read to write the chunk to a file. Im getting an out of memory error reading in the file (after about 60mb) any ideas? Have a feeling that there is some zombie references sitting about inside of my reading loop.
Also are there any good mechanisms for checking memory usage?

Cheers

Tom
 
Ernest Friedman-Hill
author and iconoclast
Marshal
Pie
Posts: 24212
35
Chrome Eclipse IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
A proper read loop should have constant memory use, so yes, there's probably a leak.

Traditionally, commercial tools like JProbe or OptimizeIt have been the best way to monitor and examine memory use; I understand there are some useful monitoring tools that come with Java 5, however.
 
Tom Hill
Ranch Hand
Posts: 115
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The out of memory error was being caused by an ObjectInputStream.
The chunks Im reading are being put into a Transfer object that contains some other useful information. I was using an ObjectIn/OutputStream to write the transfer objects between sever and client. I have discovered that references are not being updated. Calling reset causes various Exceptions. Is there an easier mechanism for transfering objects that may change during the reading file loop?
 
Tom Hill
Ranch Hand
Posts: 115
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
...because Im calling reset on the wrong stream at the wrong time!

Cheers guys, once again youve given me lots to think about!



Tom
 
Layne Lund
Ranch Hand
Posts: 3061
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you still need help, please post some code to show us what you are doing. Paraphrasing often loses important details that we need to help you.

Layne
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic