Win a copy of Murach's Python Programming this week in the Jython/Python forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Pls Help on read the log file BACKWARd?  RSS feed

 
jay lai
Ranch Hand
Posts: 180
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Please help:

I have search the forum for similar posted, but not getting the useful hints. I have a log file, and need to read backward with certain time range and only display back 1000 rows from the log.

Sample of log file


So if the user put in the time range, I will go and read the file searching from bottom up with the time range they just specified, and display back 1000 rows per request.

Thank you in advance for any helps or hints.
Regards...
 
Edwin Dalorzo
Ranch Hand
Posts: 961
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, if all the lines have the same size, it is pretty easy.

Divide the size of the file by the size of every record (line) and that will tell you how many lines the log file has. If the file has more than a 1000 lines, then calculate what is the position of the first line you should read which is: total number of lines - 1000. Then calculate the position of the first line you must read (i.e. record size * line number); otherwise the file has less than a 1000 lines and you must read it all.

Use a java.io.RandomAccessFile to seek that position and start reading from there on.

If the lines in the file are of different sizes, then you cannot foresee the number of lines in the file; therefore you will irredeemable have to read the whole file and create a buffer of a thounsand lines that you will ultimately return with the last thousand or less lines that you found after reading all the lines in the log file.
[ December 15, 2006: Message edited by: Edwin Dalorzo ]
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
One solution would be to read from the front but keep the most recent n lines in a buffer.

Now the array has the most recent BUFFER_SIZE lines, starting at buffer[(lineNumber+1)%BUFFER_SIZE]. That's easy, but could be slow if you want the last lines out of millions.

If the file is really huge, you might actually read backwards. That would be great fun but a fair bit of work to get right. A generic backwards reader could use RandomAccessFile to read a big chunk from the end of the file into a byte array, then iterate backwards through the array for a newline character.

Did either of those sound right for your problem?
 
jay lai
Ranch Hand
Posts: 180
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you much for your help, I will try out Stan James solution and see how it works. But again thanks for both solutions from Edwin Dalorzo and Stan James.

Regards.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!