• Post Reply Bookmark Topic Watch Topic
  • New Topic

Reading file from bottom in a memory efficient way

 
Aniruddh Joshi
Ranch Hand
Posts: 275
Eclipse IDE jQuery Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,
I have a file whose maximum size can reach 20Mbs, I have to parse data from this file starting from the end.
At present I save it in a StringBuffer, and iterate through it.
In each iteration I read the last 2-3 lines, save the data if it is useful and cut the read part away.
Can this be done in a more memory efficient way ?
 
Aniruddh Joshi
Ranch Hand
Posts: 275
Eclipse IDE jQuery Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why is this closed ?
 
Joanne Neal
Rancher
Posts: 3742
16
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Aniruddh Joshi wrote:Why is this closed ?


What makes you think it's closed ?
If it was closed, you wouldn't have been able to post that message.
 
Aniruddh Joshi
Ranch Hand
Posts: 275
Eclipse IDE jQuery Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
the icon on the forum isn't yellow
 
Rob Spoor
Sheriff
Posts: 20817
68
Chrome Eclipse IDE Java Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Moving to the I/O forum.
 
Henry Wong
author
Sheriff
Posts: 22516
109
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
At present I save it in a StringBuffer, and iterate through it.
In each iteration I read the last 2-3 lines, save the data if it is useful and cut the read part away.
Can this be done in a more memory efficient way ?


You need to explain this a bit more -- as we know you are *not* using the string buffer iterators. So, what do you mean by iterate though a string buffer.

Henry
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13078
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
IF I wanted to look at the last few lines of a file, I would:
1. use a RandomAccessFile method to read the last few K of characters into a byte array
2. locate the first line break in the byte array
3. create a String from the byte array starting right after the line break.
4. proceed with normal String methods.


Bill
 
Aniruddh Joshi
Ranch Hand
Posts: 275
Eclipse IDE jQuery Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Henry,

I have to read log messages from a log file, my algo follows:
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!