want to search a string within a file size 2GB

[Nagendra Ramu]

hi,
I have three files called first_err.log, first.log, first.out. In all the three files i want to get the exception names like NullPointerException, IndexOutofBounds, etc and count the total number of occurancy for each exceptions and put in a new file called Excepiton.txt.
We should use java version 1.4.x only.

please help in this. Thank u

[Joe Ess]

Welcome to the JavaRanch, Nagendra.
What in particular do you have a question about? If you are unfamiliar with working with files, The Java Tutorial has an introduction to Java IO here. If you want to get fancy, there's a number of string searching algorithms to consider once you've figured out the reading and writing part of your program.

[Nagendra Ramu]

hi
i want the code related to my question which i had mentioned earlier.
presently i am running batch file to search the exceptions , it takes lot of time. so, please give me some code related to this.
thank u

[Joe Ess]

Originally posted by Nagendra Ramu:

i want the code related to my question which i had mentioned earlier.

Ah, then you want to hire a programmer.
You see, we're all volunteers here, and while we love to help people, we insist that you Show Some Effort.

[Joe Danahoo]

If you haven't already, no need to hire a programmer, unless of course you do want it done professionally.

If you could give me a list of all the expceptions you want to parse from the files, i could code up a draft and see if it fits your needs...

Edit:

Forgive me if it doesnt work, and let me know, ill fix it.
Also, i am a beginner, just learning, so sorry if its not top-notch code

  
import java.io.*;
 
class ExceptionParser
{
public static void main(String[] args)
{
int nullPointEx, indexOOB;
string line = null;
String[] fileChoice = {"first err.log", "first.log", "first.out"}; 
 
try
{
for(x=0, x<3, x++)
{
FileReader fileReader = new FileReader(fileChoice[x]);
BufferedReader reader = new BufferedReader(fileReader);

while((line = reader.readLine()) != null)
{
if(reader.readLine() = "NullPointerException")
{
nullPointEx = nullPointEx + 1;
}
else if(reader.readLine() = "IndexOutofBounds")
{
indexOOB = indexOOB + 1;
}
}
}
}
catch(Exception ex)
{
ex.printStackTrace();
}

try
{
FileWriter writer = new FileWriter("Exceptions.txt");
 
writer.write("Amount of Exceptions in files...");
writer.write("");
writer.write("NullPointerExceptions: " + nullPointEx);
writer.write("IndexOutofBounds: " + indexOOB);
 
writer.close();
}
catch(Exception ex)
{
ex.printStackTrace();
}
}
}  

[ July 21, 2006: Message edited by: Joe Danahoo ]
[ July 21, 2006: Message edited by: Joe Danahoo ]

[Scott Cook]

Your code looks great, Joe, but perhaps you didn't notice that he said his files were 2G in size? BufferedReader will output each line of text to a String. The problem with that is that it goes into the String Pool, where it can never be deleted (except by exiting the program). That means 2G of memory will be taken up (minus the occasional line duplicates), just by reading in the files!

I mainly mention this because I'm having the same problem (trying to read-in and parse HUGE text files). A BufferedReader that returned type StringBuilder instead of String would be great, but I don't see it in the API.

Anyone have any more ideas?

Thanks!
Scott
[ August 02, 2006: Message edited by: Scott Cook ]

[Joe Ess]

Originally posted by Scott Cook:
The problem with that is that it goes into the String Pool, where it can never be deleted (except by exiting the program).

welcome to the JavaRanch, Scott
If that were true, I don't think any java program would be able to run for any amount of time without running out of memory.
Strings which are hardcoded in your program go into a literal pool and are never destroyed. Strings which are created in the course of a program's execution are elegible for garbage collection just like every other object.
[ August 02, 2006: Message edited by: Joe Ess ]

[Scott Cook]

Strings which are created in the course of a program's execution are elegible for garbage collection

Thanks for clearing that up! I read about String Immutability in the appendix of Head First Java and got really nervous. It sounded like a HUGE memory leak to me!(not literally, but has the same effect)

Now I feel much safer using BufferedReader.

[Appu Chan]

Seen another thread on this...
searching large file