I want to replace a complex string using regex.

[Kundan Dee]

Hello guys,

I want to replace

Logger.info(className, dasd + "sadsd" );


with

Logger.info(className, dasd + "sadsd" + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() + "]");


using regex.

The dasd + "sadsd" part is random and there can be any text there, we print any log message there.

How can I do it?

There are 1000s of loggers in our code that I want to replace to add the Thread line....

[Ron McLeod]

Are you asking how you update your source code?

If you are using an IDE for your development work, it most likely has a search/replace capability to do what you want.

[Kundan Dee]

I can't do search and replace for 1000s of unique strings in my ide that's why I turn to regex.

[Kundan Dee]

Ron McLeod wrote:Are you asking how you update your source code?

If you are using an IDE for your development work, it most likely has a search/replace capability to do what you want.

I can't do search and replace for 1000s of unique strings in my ide that's why I turn to regex

[Ron McLeod]

Kundan Dee wrote:I can't do search and replace for 1000s of unique strings in my ide that's why I turn to regex

Which IDE?

[Kundan Dee]

Ron McLeod wrote:

Kundan Dee wrote:I can't do search and replace for 1000s of unique strings in my ide that's why I turn to regex

Which IDE?

Eclipse for java

[Carey Brown]

	static void replace()
	{
		String	str = "Logger.info(className,dasd + \"sadsd\" );";
		Pattern pat = Pattern.compile( "Logger.info\\((.*?)\\);" );
		String s =  "";
		Matcher m = pat.matcher( str );
		if( m.matches() )
		{
			s = m.replaceFirst( "Logger.info(" + m.group(1) + " + \"[\" + "
					+ "Thread.currentThread().getStackTrace()[1].getLineNumber() + \"]\"" + " );" );
		}
		System.out.println( s );
	}

[Tim Holloway]

Welcome to the Ranch, Kundan!

You didn't actually say which of the several popular loggers you're using, but it's a general question, so I'll answer it in general form.

First of all, the "className" part of the Logger.info() isn't actually a class name, it's a logger name. Typically the name of the logger for that class is its fully-qualified class name, but as an example, Hibernate also uses logger names that cover a particular function over multiple classes, so the logger might be named something like "org.hibernate.sql".

Secondly, I think some loggers at least can print the thread ID automatically if you set up the appropriate options in their log output channels ("appenders" in log4j). So check. You might not need to alter statements. They will also report the line number of the log statement, again, if you have the proper log format string set up.

I recommend this over custom mods to the log method calls. It's far less work, and overall a cleaner and more flexible solution.

Thirdly, if you absolutely positively MUST do a global search-and-replace over a source code subtree, the Eclipse IDE can do that, I'm fairly sure that IntelliJ can do that (although it's not as obvious on how to do it) and NetBeans probably as well.

Although there are times when I just brute-force it using a quick-and-dirty Perl script or something like that.

[Ron McLeod]

With Eclipse you can search and replace (using regex) in files doing something like this.







Regular expression: Logger.info\(className, (.*) \+ "(.*)" \);
Replacement: Logger.info(className, $1 + "$2" + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() + "]");

[Kundan Dee]

Tim Holloway wrote:Welcome to the Ranch, Kundan!

You didn't actually say which of the several popular loggers you're using, but it's a general question, so I'll answer it in general form.

First of all, the "className" part of the Logger.info() isn't actually a class name, it's a logger name. Typically the name of the logger for that class is its fully-qualified class name, but as an example, Hibernate also uses logger names that cover a particular function over multiple classes, so the logger might be named something like "org.hibernate.sql".

Secondly, I think some loggers at least can print the thread ID automatically if you set up the appropriate options in their log output channels ("appenders" in log4j). So check. You might not need to alter statements. They will also report the line number of the log statement, again, if you have the proper log format string set up.

Thirdly, if you absolutely positively MUST do a global search-and-replace over a source code subtree, the Eclipse IDE can do that.

Although there are times when I just brute-force it using a quick-and-dirty Perl script or something like that.

The className is for each and every class where we are using the logos which is basically every class.

Also we are using a custom logger class so don't think about log4j or anything else.

I have given you the logger in the way that it is implemented.

Also they I need to do this for thousands of logers okay so regex is only the way and even if there was a search and replace I still need to use a regex statement so please focus your answer on that I am not able to create an appropriate regex string to pattern match

[Tim Holloway]

NIH = Not Invented Here.

The problem with roll-your-own subsystems is that they are rarely as powerful, flexible, or well-debugged as industry-standard ones. Documentation and support is usually vastly inferior, since whoever developed it probably is usually busy with "more important" stuff, if they haven't simply left the originization several years ago.

It just doesn't pay (literally $$$ or lahks, or whatever) just to demonstrate cleverness. Better to devote effort to things more specific to the actual business.

OK, my opinionating is done. As Ron's screenshot plainly shows, you CAN select a set of files within Eclipse for bulk search-and-replace and not only that, it can preview the operations so that you don't accidentally damage something.

[Kundan Dee]

Ron McLeod wrote:With Eclipse you can search and replace (using regex) in files doing something like this.







Regular expression: Logger.info\(className, (.*) \+ "(.*)" \);
Replacement: Logger.info(className, $1 + "$2" + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() + "]");

THanks a lot for your reply.

I had only given a sample logger, in reality there can be any number of strings (groups) containing many different character.

Can you club all that into a single group? A single group would be better.

Thanks

[Tim Holloway]

You can do all sorts of things with regexes. The question is whether you should.

I've used this Eclipse feature, and I find it better to make several small regex changes rather than one does-everything change. Regexes can be treacherous and it allows me to limit damage and roll it back while keeping the earlier changes that worked OK.

[Ron McLeod]

Something else to consider — when you call the logger like this:

Logger.info(className, dasd + "sadsd" + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() + "]");

Thread.currentThread().getStackTrace()[1].getLineNumber() will be called regardless of what level you have the logging set to (including higher levels like: warn, error, fatal), and that kind of call will be fairly expensive performance-wise.

One way to deal with that would be to check if logging at the wanted level is enabled before trying to get the line number:

if (Logger.isInfoEnabled()) {
	Logger.info(className, 
		dasd 
		+ "sadsd" 
		+ " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
		+ "]"
		);
}

Another, if the Logger supports it, would be to provide a Supplier:

Logger.info(className, () -> 
		dasd 
		+ "sadsd" 
		+ " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
		+ "]"
	);

[Ron McLeod]

Kundan Dee wrote:Can you club all that into a single group? A single group would be better.

It might be possible, but without any specifics its impossible to answer.

[Kundan Dee]

Ron McLeod wrote:

Kundan Dee wrote:Can you club all that into a single group? A single group would be better.

It might be possible, but without any specifics its impossible to answer.

I think i got it to work

Search: Logger.([a-z]+)\(([a-z]+),((.*)+)\);

Replace with:
Logger.$1($2, $3 + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() + "]");


Please feel free to suggest any improvements!

[Kundan Dee]

Tim Holloway wrote:You can do all sorts of things with regexes. The question is whether you should.

I've used this Eclipse feature, and I find it better to make several small regex changes rather than one does-everything change. Regexes can be treacherous and it allows me to limit damage and roll it back while keeping the earlier changes that worked OK.

Thanks Tim, I agree about the damage part.

I think i got it to work

Search: Logger.([a-z]+)\(([a-z]+),((.*)+)\);

Replace with:
Logger.$1($2, $3 + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() + "]");


Please feel free to suggest any improvements!

[Carey Brown]

The "((.*)+)" probably should be just "(.+)"

[Kundan Dee]

Carey Brown wrote:The "((.*)+)" probably should be just "(.+)"

May i know why? I'm new to this..

Both are giving me the same number of results in my code base

[Carey Brown]

".*" means zero or more of any character
"(.*)+" then would mean one or more of ( zero or more of any character )

Presuming you are using the "+" to mean that something has to be there, then
".+" does that

[Ron McLeod]

Kundan Dee wrote:I think i got it to work ... Please feel free to suggest any improvements!

Well, you haven't provided any real data to work with, so that makes it difficult to offer much of an opinion.

I did take your regular expression: Logger.([a-z]+)\(([a-z]+),((.*)+)\);
and tested it against the only sample data you have shared: Logger.info(className, dasd + "sadsd" );
and it failed to match.

I used the RegexPlanet online tool to check:

[Kundan Dee]

Ron McLeod wrote:

Kundan Dee wrote:I think i got it to work ... Please feel free to suggest any improvements!

Well, you haven't provided any real data to work with, so that makes it difficult to offer much of an opinion.

I did take your regular expression: Logger.([a-z]+)\(([a-z]+),((.*)+)\);
and tested it against the only sample data you have shared: Logger.info(className, dasd + "sadsd" );
and it failed to match.

I used the RegexPlanet online tool to check:

Hello, thanks for the reply.

In my eclipse IDE, the pattern works.


How to upload pic from local machine? I can't upload the image on the net. Those websites are blocked on my work laptop.

[Kundan Dee]

Ron McLeod wrote:
Thread.currentThread().getStackTrace()[1].getLineNumber() will be called regardless of what level you have the logging set to

Why is that?

[Carey Brown]

Ron McLeod wrote:Something else to consider — when you call the logger like this:

Logger.info(className, dasd + "sadsd" + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() + "]");

Thread.currentThread().getStackTrace()[1].getLineNumber() will be called regardless of what level you have the logging set to (including higher levels like: warn, error, fatal), and that kind of call will be fairly expensive performance-wise.

One way to deal with that would be to check if logging at the wanted level is enabled before trying to get the line number:

if (Logger.isInfoEnabled()) {
	Logger.info(className, 
		dasd 
		+ "sadsd" 
		+ " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
		+ "]"
		);
}

Another, if the Logger supports it, would be to provide a Supplier:

Logger.info(className, () -> 
		dasd 
		+ "sadsd" 
		+ " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
		+ "]"
	);

WARNING! DANGER WILL ROBINSON!
If you do a global replace and don't address this issue you'll bring your whole app to its knees.

[Kundan Dee]

Carey Brown wrote:

Ron McLeod wrote:Something else to consider — when you call the logger like this:

Logger.info(className, dasd + "sadsd" + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() + "]");

Thread.currentThread().getStackTrace()[1].getLineNumber() will be called regardless of what level you have the logging set to (including higher levels like: warn, error, fatal), and that kind of call will be fairly expensive performance-wise.

One way to deal with that would be to check if logging at the wanted level is enabled before trying to get the line number:

if (Logger.isInfoEnabled()) {
	Logger.info(className, 
		dasd 
		+ "sadsd" 
		+ " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
		+ "]"
		);
}

Another, if the Logger supports it, would be to provide a Supplier:

Logger.info(className, () -> 
		dasd 
		+ "sadsd" 
		+ " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
		+ "]"
	);

WARNING! DANGER WILL ROBINSON!
If you do a global replace and don't address this issue you'll bring your whole app to its knees.

Thanks for the warning, now I cant move forward unless I understand this issue.

I'm new to java, can you please tell me why Thread.currentThread().getStackTrace()[1].getLineNumber()  will run regardless of log level value.

Also, how does using suppliers help?

Also, how can we mention other users in our post? I want to mention @Ron McLeod

[Ron McLeod]

Kundan Dee wrote:Why is that?

Because the Logger.info method will be called regardless of the current logging level AND the argument expressions will evaluated (which includes calling Thread.currentThread().getStackTrace()[1].getLineNumber()) before calling the Logger.info method.

[Carey Brown]

Ron McLeod wrote:
Another, if the Logger supports it, would be to provide a Supplier:

Logger.info(className, () -> 
		dasd 
		+ "sadsd" 
		+ " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
		+ "]"
	);

It took me a minute to catch on but this is cool. Seeing as how the Logger is invented-here a not-too-difficult edit could support Supplier. The advantage I see of this is that the global replace we're contemplating here could easily insert this Supplier in place with no additional edits needed.

[Kundan Dee]

Ron McLeod wrote:

Kundan Dee wrote:Why is that?

Because the Logger.info method will be called regardless of the current logging level AND the argument expressions will evaluated (which includes calling Thread.currentThread().getStackTrace()[1].getLineNumber()) before calling the Logger.info method.

Our Custom logger has 3 levels... INFO, DEBUG and ERROR

When we write Logger.info(className, "Log here") => This is info log

When we write Logger.error(className, "Log here") => This is error log

When we write Logger.debug(className, "Log here") => This is debug log

Does this new info change anything?

[Carey Brown]

If you use Supplier, exactly what line number will you get?

[Kundan Dee]

Carey Brown wrote:If you use Supplier, exactly what line number will you get?

I don't understand the question.

I'll get the same line as before from where I'm calling the logger.

[Carey Brown]

Kundan Dee wrote:

Carey Brown wrote:If you use Supplier, exactly what line number will you get?

I don't understand the question.

I'll get the same line as before from where I'm calling the logger.

If you pass in a Supplier but the  Supplier code isn't executed until it arrives in Logger.info(), then what?

I wrote a little test program and it worked as it should.

	static void logInfo( String className, Supplier<Integer> sup )
	{
		System.out.printf( "INFO: %s, Line: %d%n", className, sup.get() );
	}
	
	static void logit()
	{
        // the following line number is logged
		logInfo( "className", () -> Thread.currentThread().getStackTrace()[1].getLineNumber() );
	}

[Ron McLeod]

Calling getStackTrace() is a special case and you would need to be aware that using a supplier would add 2 more stack trace elements to the the stack dump.

So: Thread.currentThread().getStackTrace()[3].getLineNumber() instead of Thread.currentThread().getStackTrace()[1].getLineNumber().

Here's a quick (so-so) example showing how the expensive call would be made whether logging was enabled or not:
import java.util.function.Supplier; public class LogTest {    boolean loggingEnabled;    public static void main(String[] args) {        var lt = new LogTest();        lt.runTest(true);        lt.runTest(false);    }    void runTest(boolean isEnabled) {        loggingEnabled = isEnabled;        System.out.println("logging enabled?: " + loggingEnabled);        System.out.println("not using supplier");        logIt(getLineNumber(2));        System.out.println("using supplier");        logIt(() -> getLineNumber(4));        System.out.println();    }    int getLineNumber(int stackDepth) {        System.out.println("making expensive call");        return Thread.currentThread().getStackTrace()[stackDepth].getLineNumber();    }    void logIt(Object msg) {        if (loggingEnabled) {            writeToLog(msg.toString());        }    }    void logIt(Supplier<Object> supplier) {        if (loggingEnabled) {            var msg = supplier.get();            writeToLog(msg.toString());        }    }    void writeToLog(String msg) {        System.out.println("log: " + msg);    } }
> java .\LogTest.java logging enabled?: true not using supplier making expensive call log: 19 using supplier making expensive call log: 21 logging enabled?: false not using supplier making expensive call using supplier

Note that I added another element to the stack because I called a method named getLineNumber and not getStackTrace directly.  I did this so that I could show that a call to getStackTrace was actually made.

[Carey Brown]

Ron McLeod wrote:Calling getStackTrace() is a special case and you would need to be aware that using a supplier would add 2 more stack trace elements to the the stack dump.

So: Thread.currentThread().getStackTrace()[3].getLineNumber() instead of Thread.currentThread().getStackTrace()[1].getLineNumber().

That's what I thought but running my code snippet showed that the line number was correctly generated at the ()->... and NOT inside the logInfo method. I even traced it with a debugger and was surprised to see the logInfo() execution jump briefly back to the ()->... before returning to logInfo() to print the message. Knock me over with a feather. Learned something new.

[Kundan Dee]

Ron McLeod wrote:Calling getStackTrace() is a special case and you would need to be aware that using a supplier would add 2 more stack trace elements to the the stack dump.

So: Thread.currentThread().getStackTrace()[3].getLineNumber() instead of Thread.currentThread().getStackTrace()[1].getLineNumber().

Here's a quick (so-so) example showing how the expensive call would be made whether logging was enabled or not:
import java.util.function.Supplier; public class LogTest {    boolean loggingEnabled;    public static void main(String[] args) {        var lt = new LogTest();        lt.runTest(true);        lt.runTest(false);    }    void runTest(boolean isEnabled) {        loggingEnabled = isEnabled;        System.out.println("logging enabled?: " + loggingEnabled);        System.out.println("not using supplier");        logIt(getLineNumber(2));        System.out.println("using supplier");        logIt(() -> getLineNumber(4));        System.out.println();    }    int getLineNumber(int stackDepth) {        System.out.println("making expensive call");        return Thread.currentThread().getStackTrace()[stackDepth].getLineNumber();    }    void logIt(Object msg) {        if (loggingEnabled) {            writeToLog(msg.toString());        }    }    void logIt(Supplier<Object> supplier) {        if (loggingEnabled) {            var msg = supplier.get();            writeToLog(msg.toString());        }    }    void writeToLog(String msg) {        System.out.println("log: " + msg);    } }
> java .\LogTest.java logging enabled?: true not using supplier making expensive call log: 19 using supplier making expensive call log: 21 logging enabled?: false not using supplier making expensive call using supplier

Note that I added another element to the stack because I called a method named getLineNumber and not getStackTrace directly.  I did this so that I could show that a call to getStackTrace was actually made.

Thanks a lot for your extensive reply.

Sadly, I didnt really get the point you were trying to make.

So using suppliers is expensive and I should use them? Right?

Or I have to use them because Thread.currentThread().getStackTrace()[1].getLineNumber(); is performance heavy for some reason and using supplier would impact performance for the better?

This is new to me, thanks for your patience!

[Carey Brown]

Ron McLeod wrote:Calling getStackTrace() is a special case and you would need to be aware that using a supplier would add 2 more stack trace elements to the the stack dump.

So: Thread.currentThread().getStackTrace()[3].getLineNumber() instead of Thread.currentThread().getStackTrace()[1].getLineNumber().

An alternate approach to a global replace would be to edit the three logging methods and get the line number from [3] as you point out.

[Ron McLeod]

Kundan Dee wrote:Sadly, I didnt really get the point you were trying to make ...

The point is that if you evaluate the line number in the call to Logger.info, then getStackTrace will be always be called, whether tracing is enabled or not.

If the logger doesn't support providing a supplier, then just wrap you calls to the logger with a check to see if logged is enabled.  This example:

if (Logger.isInfoEnabled()) {
    Logger.info(className, 
        dasd 
        + "sadsd"
        + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
        + "]"
        );
}

[Carey Brown]

Ron McLeod wrote:This example:

if (Logger.isInfoEnabled()) {
    Logger.info(className, 
        dasd 
        + "sadsd"
        + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
        + "]"
        );
}

Which works perfectly well but makes the global replace very tedious.

[Kundan Dee]

Ron McLeod wrote:

Kundan Dee wrote:Sadly, I didnt really get the point you were trying to make ...

The point is that if you evaluate the line number in the call to Logger.info, then getStackTrace will be always be called, whether tracing is enabled or not.

If the logger doesn't support providing a supplier, then just wrap you calls to the logger with a check to see if logged is enabled.  This example:

if (Logger.isInfoEnabled()) {
    Logger.info(className, 
        dasd 
        + "sadsd"
        + " [" + Thread.currentThread().getStackTrace()[1].getLineNumber() 
        + "]"
        );
}

I get you now. There is some misunderstanding in how the logs work between you and me. Ours is a custom logger and maybe in your mind our logger works in a different way.

The whole point of adding like numbers is to know during debugging, which line of code is being executed. Logs will always be enabled.
Logger.info Logger.debug and Logger.error will always be called. Its for the best when a customer faces an issue , we can debug quickly.

So now are you telling me that running Thread.currentThread().getStackTrace()[1].getLineNumber()  on every log line will cripple our application?

I tried running Thread.currentThread().getStackTrace()[1].getLineNumber()  in a loop to check timings for 5000 runs and its in milli seconds which is currently acceptable.

Is there any concern of your im missing here?

Thanks

[Carey Brown]

Kundan Dee wrote:I tried running Thread.currentThread().getStackTrace()[1].getLineNumber()  in a loop to check timings for 5000 runs and its in milli seconds which is currently acceptable.

I consider "milliseconds" to be expensive, but if that's ok with you.

I always wished that Java had something like C's __LINE__ which was computed at compile time. I even thought of writing my own simple pre-processor to do that.

[Kundan Dee]

Carey Brown wrote:

Kundan Dee wrote:I tried running Thread.currentThread().getStackTrace()[1].getLineNumber()  in a loop to check timings for 5000 runs and its in milli seconds which is currently acceptable.

I consider "milliseconds" to be expensive, but if that's ok with you.

Wow man, i wish i could say that and bring the time further down

Our application is quite date intensive so operations(import, export, etc of project models) take long time so an overall addition of 1 minute is okay imo.