Win a copy of Modern JavaScript for the Impatient this week in the Server-Side JavaScript and NodeJS forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Bear Bibeault
  • Junilu Lacar
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • salvin francis
  • Frits Walraven
Bartenders:
  • Scott Selikoff
  • Piet Souris
  • Carey Brown

Writing into a file through java

 
Greenhorn
Posts: 11
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi
I am performing read and write operation in Open VMS environment by using a java jar. Reading is fast and the code is able to read 40million records in 2hrs.
But when it come to writing to a file , it initially run fast i.e. 100000 records in 2 mins.,
but after some time it run very slow i.e. 100000 records in 2 hrs. and eventully does not proceed at all.
I am using BufferOutputStream to write into the file.
Any reasons!!!
 
Rancher
Posts: 43016
76
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Welcome to JavaRanch.

Does memory consumption go up a lot, and not come down much? The described behavior would be consistent with a memory leak. Or maybe some data structures grow a lot, and are not being handled efficiently?
 
Ranch Hand
Posts: 1376
Eclipse IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Welcome to Java ranch !!
Share your File I/O code. It shall help us to see where the problem is.
 
Sidharth Dash
Greenhorn
Posts: 11
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thanks Dittmer
i am performing two steps:
1) Reading the file through BufferInputStream and kepping the value in a string.
2) Writing the string through BufferOutputStream to the file.

I have tried running step 1 only and it run perfectly fine.
but when i have included step 2 in it it has show problems.
 
Ulf Dittmer
Rancher
Posts: 43016
76
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
OK, but that doesn't answer any of the questions I asked.
 
Sidharth Dash
Greenhorn
Posts: 11
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Ulf Dittmer wrote:Welcome to JavaRanch.

Does memory consumption go up a lot, and not come down much? The described behavior would be consistent with a memory leak. Or maybe some data structures grow a lot, and are not being handled efficiently?


there is no issues with memory consumption
as i have said its a simple operation
1)reading 40mill. records and keeping one record at a time into a string . this step has no problem
but when i tried to do writing logic it show problems...


 
Ulf Dittmer
Rancher
Posts: 43016
76
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

there is no issues with memory consumption


How do you know that? How are you observing memory consumption and GC patterns over time?

as i have said its a simple operation


That may be, but somewhere along the way there seem to be a complication, no? But without seeing the code it's hard to advise what to investigate next.
 
Marshal
Posts: 70206
280
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Are you keeping one String? Are you repeatedly adding to the String with the + operator or +=?
 
Sidharth Dash
Greenhorn
Posts: 11
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Campbell Ritchie wrote:Are you keeping one String? Are you repeatedly adding to the String with the + operator or +=?


I am just assigning line after reading from file to the string.
for every line the the string will have a new value. keep in mind only one string is used and it is only assigned new values. no+ or += operators are used
 
Rancher
Posts: 989
9
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Sidharth Dash wrote:keep in mind only one string is used and it is only assigned new values. no+ or += operators are used


That is not possible. Strings are immutable so their values cannot be changed. New Strings are being created all the time. You need to monitor the application using something like jconsole (it's free and comes with the jdk) and see what issues it can show.
 
Sidharth Dash
Greenhorn
Posts: 11
Eclipse IDE Spring Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
here is the code snippet:

1)String Line="";
2)Line = methodName(new String[] { queuedInputLine })[0];
3)writeToFile(bfrOutputStream,Line.getBytes());


line 1 is declared globally

when im executing the program with line 2 (commenting out line 3) there is no issue at all for 40 mill. records.
when im including line 3 in program, the issue arises.
if memory allocation is a concern for string "line" than line 2 itself wouldnt have executed for 40 mill. records
any ideas!!!
 
Bartender
Posts: 2233
63
IntelliJ IDE Firefox Browser Spring Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
What are methods methodName and writeToFile doing? Can you post the code?
 
Ulf Dittmer
Rancher
Posts: 43016
76
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes, please post all relevant code, these bits and pieces of information don't really help us in getting the big picture.

methodName(new String[] { queuedInputLine })[0]


This looks odd. It creates a string array, but only ever passes a single string into it. And the method returns a string array, but only ever looks at the first element. It seems that the signature could be changed from String[] methodName (String[]) to String methodName (String).
 
Campbell Ritchie
Marshal
Posts: 70206
280
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Sidharth Dash wrote: . . .
line 1 is declared globally
. . .

I hope that “globally” doesn't mean globally. I suspect line can beneficially be a local variable but if it is a field it should have private access. I cannot be sure because you posted so little code.
 
Don't get me started about those stupid light bulbs.
    Bookmark Topic Watch Topic
  • New Topic