File

[naga eswar]

Hi everybody,

I need a file related requirement. I have some files which contains a word say "log.debug".

I need which searches the word "log.debug" and its should insert a condition "if(log>10) " above the line which contains "log.debug" .

below is my program which searches the "log.debug". Please help for the rest of the part

import java.io.*;

public class searchfile {
	public static void main(String args[]) {
		try {
			// Open the file c:\test.txt as a buffered reader
			BufferedReader bf = new BufferedReader(new FileReader("D:\\Eswar\\Java_Programs\\Files\\dir1\\Controller.java"));
			String logger="logger.debug";
			
			// Start a line count and declare a string to hold our current line.
			int linecount = 0;
    			String line;

			// Let the user know what we are searching for
			System.out.println("Searching for " + logger+" in file...");

			// Loop through each line, stashing the line into our line variable.
			while (( line = bf.readLine()) != null)
			{
    				// Increment the count and find the index of the word
    				linecount++;
    				int loggerindexfound = line.indexOf(logger);
					int logindexfound = line.indexOf(log);

    				// If greater than -1, means we found the word
    				if (loggerindexfound > -1) {
    				     System.out.println("Word was found at position " + loggerindexfound + " on line " + linecount);
    				}
					
			}

			// Close the file after done searching
			bf.close();
		}
		catch (IOException e) {
			System.out.println("IO Error Occurred: " + e.toString());
		}
	}
}

[Anayonkar Shivalkar]

Hi naga,

You are almost there

All you need to do is:
1) create a new file
2) read each line from original file
3) when current line contains log.debug, insert a line if(log>10) followed by current line. otherwise, simply write current line to new file. do this till the end.

To make it more efficient, you can create temp file after first occurrence of log.debug (so that if a file does not contain log.debug, then you won't load your disk unnecessarily by creating a temp file)

I hope this helps.

[naga eswar]

Thanks a lot for your reply.


I will do as you say... but is we cant update in the existing file without creating new file because i have to update this line in above 2500 files.

[Campbell Ritchie]

You could also put the lines into a List<String> and insert into the List. A LinkedList might be better than an ArrayList for insertion in the middle. I presume you would have more than enough heap memory for a few thousand lines.

[Jeff Verdegan]

naga eswar wrote:Thanks a lot for your reply.


I will do as you say... but is we cant update in the existing file without creating new file because i have to update this line in above 2500 files.

Why would a new file of 2500 lines be a problem?

But no, you can't just update a file in place and push lines further down to insert new ones. File systems don't work that way. If you want to insert in the middle of the file, you have to rewrite everything after where you're inserting. So you have two choices:

1) Use a temp file which replaces the original when you're done. The advantages of this approach are that if a problem occurs along the way, your original hasn't been touched, and that it generally can handle larger files than the second approach, since it relies on disk, not memory, which is usually more plentiful.

2) Read the whole file into memory (or at least everything after the first change), and insert lines where you need them, using a List, then write it back out, replacing the old file (or the part of it after the first insert).

[Wendy Gibbons]

Jeff Verdegan wrote:

naga eswar wrote:Thanks a lot for your reply.


I will do as you say... but is we cant update in the existing file without creating new file because i have to update this line in above 2500 files.

Why would a new file of 2500 lines be a problem?

But no, you can't just update a file in place and push lines further down to insert new ones. File systems don't work that way. If you want to insert in the middle of the file, you have to rewrite everything after where you're inserting. So you have two choices:

1) Use a temp file which replaces the original when you're done. The advantages of this approach are that if a problem occurs along the way, your original hasn't been touched, and that it generally can handle larger files than the second approach, since it relies on disk, not memory, which is usually more plentiful.

2) Read the whole file into memory (or at least everything after the first change), and insert lines where you need them, using a List, then write it back out, replacing the old file (or the part of it after the first insert).

Jeff I think he was saying he had 2500 files to change. so couldn't afford to duplicate that many files.

[Jeff Verdegan]

Wendy Gibbons wrote:
Jeff I think he was saying he had 2500 files to change. so couldn't afford to duplicate that many files.

D'OH! That's twice in less than 24 hours that I didn't properly ReadTheQuestion!

The thing is, though, regardless of which approach he takes, he will have to duplicate the 2500 files (or at least the parts after their respective first inserts). Whether he does it in place or in a temp file is an irrelevant detail, unless for some reason he has to have all 2500 "active" at once.

[Wendy Gibbons]

if he makes a temp file and then replaces the original (a thing I wouldn't do myself, but then I am a paranoid little thing) he only needs to have 2501 at max

[Jeff Verdegan]

Wendy Gibbons wrote:if he makes a temp file and then replaces the original he only needs to have 2501 at max

Right. But he still needs to copy 2500 files. Doing them one at a time means there will only exist 2501 at any given point in time, but it doesn't save him the copying.

If the concern was that there would be 2500 extra files at the same time, then I'm confused, as I didn't see any advice given that would lead to that. Or was I just not reading closely enough again?

(a thing I wouldn't do myself, but then I am a paranoid little thing)

Why not? That's safer than overwriting the original in place.

[Wendy Gibbons]

i would leave both files, incase it turned out something had gone very wrong

[naga eswar]

hi guys really thanks for discussions for me and sorry for being very late reply....


The problem to create duplicates is memory /space (for 2500 extra files).
No problem if i update existing file because the original one will lies at another system.

[Jeff Verdegan]

naga eswar wrote:
The problem to create duplicates is memory /space (for 2500 extra files).

That's only a problem if you keep all 2500 files open at once (or if any one file is extremely large). Is there any reason you need to do all 2500 at one time, instead of one after the other?

[naga eswar]

ya.. it will take lots of time if i do one by one and also if don't have the word (which I am searching) in a particular file

So i want to keep all these files in one folder.
All files has to update at once if i mention the folder path.

[Winston Gutkowski]

naga eswar wrote:ya.. it will take lots of time if i do one by one and also if don't have the word (which I am searching) in a particular file

I think it'll take a while whatever you do; but that has more to do with the fact that you have 2,500 files than anything else.

So i want to keep all these files in one folder.
All files has to update at once if i mention the folder path.

One folder, a hundred folders; it really doesn't matter to Java.

Personally, I think your problems start with a design that relies on searching 2,500 files - but if you've gotta do it, you've gotta do it; and it'll take as long as it takes.

Winston

[Jeff Verdegan]

naga eswar wrote:ya.. it will take lots of time if i do one by one

Why do you think that?

I'm not talking about time. Yes, you can probably make it run a little faster with judicious use of multithreading. But I'm talking about the task requirements. Do you have to work on all the files at the same time, due to some interdependence among them?

If not, then the only reason to do them all in parallel is if ALL of the following conditions are true:

1. The changes you're making are the same in all the files, or at least many of them.

AND

2. It is expensive to compute those changes, so you want to compute them once and them apply them to all the files at the same time.

AND

3. The changes are too large to keep in memory, so that you can't just compute them once and then apply them to all the files sequentially.

So i want to keep all these files in one folder.

What do the folders have to do with it?

All files has to update at once if i mention the folder path.

What? No. No matter what you do, the actual physical files will be updated one at a time. You may be able to make use of Java's multithreading capabilities such that at a higher level of abstraction in your progam it appears as though multiple files are being handled simultaneously, but in reality, the bytes will be written out in order. And which folder they're in has nothing to do with how fast it will go (unless different folders are on different physical disks).

[naga eswar]

Thanks for replying.

here I used wrong sentence "All files has to update at once".

If I give folder name in the program, then the files inside that folder has to update(one by one automatically).

The above program i have mentioned is only for one file at a time. So for every file i have change the path of it 2500 times.

Thats why I am saying it takes lot of time and the changes making are the same in all the files.

[Paul Clapham]

Okay. And there has been a lot of discussion in this thread. So do you still have a question? (Your latest post is not a question.) If so, what is it and why has it not been answered by all of the previous discussion here?

[naga eswar]

sorry for confused made..

The latest post from my side is thta i replied to "Jeff Verdegan "

[Paul Clapham]

naga eswar wrote:The latest post from my side is thta i replied to "Jeff Verdegan "

You have done that several times. I think it's time for you to summarize what your problem is now.

[Winston Gutkowski]

naga eswar wrote:ya.. it will take lots of time if i do one by one and also if don't have the word (which I am searching) in a particular file

Well, if you're on Unix, one possibility is to run a 'grep' on all the files first and then use Java to do the updates. For example:

grep -n "log.debug" * > edit.data

would read all the files in a directory and produce output like:
file1.dat:23: log.debug edited on yyyy/mm/dd
file1.dat:49: log.debug edited on yyyy/mm/dd
file2.dat:373: log.debug edited on yyyy/mm/dd
...
where the first field is the file name, the 2nd the line number, and the third the actual data.
You could then run that data through a Java program, which would then only edit the files in which your string was found.
In fact, you could pipe the output through a 'cut' to produce only the filenames and line numbers.

But that's on Unix. On Windows....

Winston

[naga eswar]

ok Paul...

here my problem is that i have logger method(ex. debug) which we are using in maxmium files of our application.

Due to memory related issues we need to insert condition before the method executes. So i need to insert a sentence "if(isDevelopmentEnabled()" before every log.debug(""); method. So first i need to search the word log.debug and if founds "if(isDevelopmentEnabled()" should insert before that line. we have around 2500 files in my application. So its too difficult to do manually. Thats why I need in program to search and update all files.

I have started this with no idea. Thats I have confused you. Really sorry.

My requirement is that.... I need to give the folder path which the files contains. It has to search all the files and if founds the word, it should add "if(isDevelopmentEnabled()" should insert before that line where the searching word exists.

Please help. Let me know if anything required

[Joanne Neal]

You were told how to update the files in the very first reply.
As far as finding all the files in a given directory, the File class has methods to do that.

[naga eswar]

Thanks Joanne for your reply.

Can you please elobarate which methods we can use and how.

[Joanne Neal]

naga eswar wrote:Can you please elobarate which methods we can use and how.

Are you seriously saying you've looked at the Javadoc for the File class and can't find any methods that will list all the files in a directory ?

[Jeff Verdegan]

naga eswar wrote:ok Paul...

here my problem is that i have logger method(ex. debug) which we are using in maxmium files of our application.

Due to memory related issues we need to insert condition before the method executes. So i need to insert a sentence "if(isDevelopmentEnabled()" before every log.debug(""); method. So first i need to search the word log.debug and if founds "if(isDevelopmentEnabled()" should insert before that line. we have around 2500 files in my application. So its too difficult to do manually. Thats why I need in program to search and update all files.

There's no reason to write a program to do this. Just open an IDE or any decent programmer's editor and use its "replace all in directory" (or "...in project" feature.

[naga eswar]

Hi Jeff/Joanne... Thanks for reply....

@Jeff i am not doing replace. i am trying to insert the condition.

If such editor(I am using Eclipse) is there please let me know.


@Joanne... listing the files in a directory is ok... but my requirement is to edit that each file in that directory...

[naga eswar]

Hi every body... some what i got . I can search the required word in all the files in directory and its sub directiry

Below is the program

import java.io.*; 
import java.util.Date;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Calendar;

public class Listing {    
 
 static int count=0;  
 static int spc_count=-1; 
  static int x=0; 
  
  static int Process(File aFile) {   
  spc_count++;    
  String spcs = ""; 
  String fileName=null;

  for (int i = 0; i < spc_count; i++)  
  spcs += " ";   

  if(aFile.isFile()){   
  //System.out.println(spcs + "[FILE] " + aFile.getName());
     fileName=  aFile.getPath();
	 
	 x=searchFile(fileName);
	 
	}
  
  else if (aFile.isDirectory()) { 
  
    //  System.out.println(spcs + "[DIR] " + aFile.getName());  
      File[] listOfFiles = aFile.listFiles();  
      
	  if(listOfFiles!=null) {  
	  
       for (int i = 0; i < listOfFiles.length; i++)      
       Process(listOfFiles[i]);        
  } else {      
  System.out.println(spcs + " [ACCESS DENIED]");    
  }  
  }    
  spc_count--; 
  return x;
   
  }      
  
  public static void main(String[] args) {     

  String nam = "C:\\MNYL\\src\\com\\mastek\\eElixir";    
  long time;
  File aFile = new File(nam); 
  time = System.currentTimeMillis();
  int y=Process(aFile);
  time = System.currentTimeMillis() - time;
  
  int seconds = (int) (time / 1000) % 60 ; 
  int minutes = (int) ((time / (1000*60)) % 60); 
  int hours   = (int) ((time / (1000*60*60)) % 24);  
  
  System.out.println(" Total Searches:  " + y);  
  System.out.println(" Time taken is  " + hours +":"+minutes+":"+seconds);  
  }


  static int searchFile(String filename){
         
  
       try {
	         
			 			// Open the file c:\test.txt as a buffered reader
			FileReader fr=new FileReader(filename);
			BufferedReader bf = new BufferedReader(fr);
			
			// Start a line count and declare a string to hold our current line.
			String logger="logger.debug";
			String log="log.debug";
			
			int linecount = 0;
    			String line;

			// Let the user know what we are searching for
			//System.out.println("Searching for " + logger +" & "+ log + " in file...");

			// Loop through each line, stashing the line into our line variable.
			while (( line = bf.readLine()) != null)
			{
    				// Increment the count and find the index of the word
    				linecount++;
    				int indexlogger = line.indexOf(logger);
					int indexlog = line.indexOf(log);

    				// If greater than -1, means we found the word
    				if (indexlogger > -1) {
    				   //  System.out.println(logger +"  " + indexlogger + "  " + linecount + "  "+ filename);
						 ++count;
    				}
					if (indexlog > -1) {
    				 //    System.out.println(log +"  " + indexlog + "  " + linecount + "  "+ filename);
						 ++count;
    				}
			}
			
			

			// Close the file after done searching
			bf.close();
			
		}
		catch (IOException e) {
			System.out.println("IO Error Occurred: " + e.toString());
		}
       return count;
  }
  }

[Joanne Neal]

naga eswar wrote:I can search the required word in all the files in directory and its sub directiry

Okay. So now all you have to do is go back to Jeff Verdegan's first post and follow one of his suggestions there.
Your only objection to option 1 that I can see is that you thought it would create 2500 temporary files. As long as you handle one file at a time (which you are doing) then you only need one temp file. Just do the following for each source file

Read the source file line by line.
if you find one of the logging lines
    write out the required extra line to the temp file
write out the line you read to the temp file
when complete, delete the original file and rename the temp file

[Jeff Verdegan]

naga eswar wrote:
@Jeff i am not doing replace. i am trying to insert the condition.

It's the same thing. And yes, you are doing a replace. You're replacing debug(x) with if (condition) debug(x) or something along those lines.

If such editor(I am using Eclipse) is there please let me know.

I haven't used Eclipse for a long time, but I'm sure it has it. Look for something like global search and replace or project search and replace in its menus or docs. (I'm sure it has a search function in its documentation.)

@Joanne... listing the files in a directory is ok... but my requirement is to edit that each file in that directory...

Yes. But how are you going to edit each one if you don't list them first? Note that "list" here doesn't necessarily mean "display a list of files." In this context, listing them means getting that list of files.

[naga eswar]

Thanks a lot for replying guys.

i am working as you said .... I get back with code once i finished.

[Jeff Verdegan]

naga eswar wrote:Thanks a lot for replying guys.

i am working as you said .... I get back with code once i finished.

So you've decided not to just use Eclipse for this after all?

[naga eswar]

Hi jeff its like that...

if i work on code, i can learn a lot on java.
surely i will work on eclipse. So i can learn both.
really guys i am learning lot here from you all.
Thanks a lot.

[naga eswar]

Hi every body... after all banging my head against the wall I got it

I heartfully thanks for every one who helped me. Below is the code.

If any suggestions you are welcome.

import java.io.*; 
import java.util.*;
import java.text.*;

public class editFiles {  
      
static void Process(File aFile) {   
            String spcs = ""; 
            String fileName=null;
  
    if(aFile.isFile()){  
       fileName=  aFile.getPath();
       if(fileName.endsWith(".java"))	   
	   searchFile(fileName);	 
	}
  
  else if (aFile.isDirectory()) { 
      File[] listOfFiles = aFile.listFiles();  

	  if(listOfFiles!=null) {  
       for (int i = 0; i < listOfFiles.length; i++)      
          Process(listOfFiles[i]);        
  } 
  else {      
  System.out.println(spcs + " [ACCESS DENIED]");    
   }  
 }    
}      
  
public static void main(String[] args) {     

    String nam = "D:\\log.java";    
    long time;
    int y;
    
	File aFile = new File(nam);
    time = System.currentTimeMillis();
    Process(aFile);
   
    time = System.currentTimeMillis() - time;
  
    int seconds = (int) (time / 1000) % 60 ; 
    int minutes = (int) ((time / (1000*60)) % 60); 
    int hours   = (int) ((time / (1000*60*60)) % 24);  
  
    System.out.println(" Time taken is  " + hours +":"+minutes+":"+seconds);  
  }

static void searchFile(String filename){
  
        File f=new File(filename);
		System.out.println(f);
	
        FileInputStream fs = null;
        InputStreamReader in = null;
        BufferedReader br = null;
		BufferedWriter outobj=null;
		
       	LinkedList ll=new LinkedList();
		int lllenght=0;
        String textLine;
        StringBuffer textinLine=new StringBuffer("");
		String ifLine="\n"+"          if(ok)"+"\n";
		String lineLog="log.debug";
		         
        try {
             fs = new FileInputStream(f);
             in = new InputStreamReader(fs);
             br = new BufferedReader(in);

            while(true)
            {
                textLine=br.readLine();
                if(textLine==null)
                    break;
				
				if(textLine.contains(lineLog)){
				
				      ll.add(ifLine);
					}
                ll.add(textLine+"\n");
			}
			
			   lllenght=ll.size();
			            
              fs.close();
              in.close();
              br.close();

            } catch (FileNotFoundException e) {
              e.printStackTrace();
            } catch (IOException e) {
              e.printStackTrace();
            }

            try{
			
			     for(int k=0;k<lllenght;k++){
				 
				 textinLine.append(ll.get(k));
				 int bbl=textinLine.length();
                 FileWriter fstream = new FileWriter(f);
                outobj = new BufferedWriter(fstream);
                outobj.write(textinLine.toString());
				}
			}
              catch (Exception e){
              System.err.println("Error: " + e.getMessage());
           }
		   finally{
		      try{
		        outobj.flush();
				outobj.close();
				}catch (Exception e){
              System.err.println("Error: " + e.getMessage());
           }
				
		}
		   
      }
  }

[Campbell Ritchie]

Why are you closing some objects in a finally and some not? Why are you using flush() before close()?

[naga eswar]

I didnot observed much about closing of objects I observed only output.

I have to work on it again. Regarding flush the some code was brought from google search. I think it was remained same there.
I observed only logic and java methods using. Thanks for information.

[Campbell Ritchie]

Read the API for close() and you find out flush is unnecessary.

[naga eswar]

Campbell Ritchie wrote:Read the API for close() and you find out flush is unnecessary.

Hi Campbell, I have already removed the out flush and closed all objects in finally. thanks for updating..