• Post Reply Bookmark Topic Watch Topic
  • New Topic

The file size problem

 
Hugo Leung
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If i think that the file too large size(~90MB) , i can not use the BufferedReader,
so what should i do ?
 
Vlado Zajac
Ranch Hand
Posts: 245
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There is no such thing as too large file for BufferedReader.

It depends on how you work with the data but BufferedReader itself has not problem with any input size.
 
Deepak Bala
Bartender
Posts: 6663
5
Firefox Browser Linux MyEclipse IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So what are you doing with the file and what error occurred ? Can you post some code and a stack trace ?
 
Ranganath Junpal
Ranch Hand
Posts: 76
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well i have the similiar problem.There is huge amount of data to be copied. Here is the code which i did for it.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//copy the contents of those files..
public File createFile(File classFolder,File Ofile)
{
File file=new File(classFolder,Ofile.getName());
file.deleteOnExit();
BufferedInputStream inStream=null;
BufferedOutputStream outStream=null;
PrintStream oStream=null;
DataInputStream dStream =null;
try{
inStream=new BufferedInputStream(new FileInputStream(Ofile));
outStream=new BufferedOutputStream(new FileOutputStream(file));
oStream=new PrintStream(outStream);
dStream=new DataInputStream(inStream);

while (dStream.available() != 0)
{
// this statement reads the line from the file and print it to
// the console.
PrintStream out=new PrintStream(System.out);
System.setOut(oStream);
System.out.println(dStream.readLine());
System.setOut(out);
}
inStream.close();
outStream.close();
oStream.close();
dStream.close();
}
catch(IOException ioException)
{

JOptionPane.showMessageDialog(null,"The project Couldnt be Opened. " +
"Its corrupted","Code Companion",JOptionPane.ERROR_MESSAGE);
ioException.printStackTrace();
}

return file;
}

----------------------------------------------------------------------------

This works fine to copy 10-15 files after that it gives exception as follows.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
xception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space
at java.io.BufferedWriter.<init>(BufferedWriter.java:87)
at java.io.BufferedWriter.<init>(BufferedWriter.java:70)
at java.io.PrintStream.init(PrintStream.java:83)
at java.io.PrintStream.<init>(PrintStream.java:100)
at java.io.PrintStream.<init>(PrintStream.java:62)
at com.jenssoft.codecompanion.gui.ProjectJTree.createFile(ProjectJTree.java:95)
at com.jenssoft.codecompanion.gui.ProjectJTree.bufferProjectFiles(ProjectJTree.java:71)
at com.jenssoft.codecompanion.gui.ProjectJTree.storeInfo(ProjectJTree.java:125)
at com.jenssoft.codecompanion.gui.ProjectJTree.<init>(ProjectJTree.java:42)
at com.jenssoft.codecompanion.gui.MainFrame.actionPerformed(MainFrame.java:455)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1995)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2318)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.AbstractButton.doClick(AbstractButton.java:357)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:1170)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:1211)
at java.awt.Component.processMouseEvent(Component.java:6038)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3260)
at java.awt.Component.processEvent(Component.java:5803)
at java.awt.Container.processEvent(Container.java:2058)
at java.awt.Component.dispatchEventImpl(Component.java:4410)
at java.awt.Container.dispatchEventImpl(Container.java:2116)
at java.awt.Component.dispatchEvent(Component.java:4240)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4322)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:3986)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:3916)
at java.awt.Container.dispatchEventImpl(Container.java:2102)
at java.awt.Window.dispatchEventImpl(Window.java:2429)
at java.awt.Component.dispatchEvent(Component.java:4240)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:273)

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

i am sure there is something wrong here.. so please help me out like wht needs to be done.

Thank you
 
Joe Ess
Bartender
Posts: 9362
11
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
One problem that I see, Ranganath, is that Available doesn't do what you think it does. The other problem is that you keep creating instances of PrintStream for System.out. Since you do this with each iteration in the loop, eventually you run out of memory. It is not necessary to create a print stream for System.out, It already exists. Have a look at this example.
As for Hugo's problem, if you are attempting to read the entire file into memory, you will have problems. The default VM size is 64 megs, so you will not be able to read in a file half that size. Your options are to make the VM larger (use the -Xmx option on the java command line) or process your file in smaller pieces.
[ March 19, 2007: Message edited by: Joe Ess ]
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!