Logging Vs Performance

[Nikhil Joshi]

Hi,

We are using a Logging Utility much similar to LOG4j.

We are logging some information at DEBUG,INFO,ERROR levels.

However, I observed that in case of multiple concurrent requests, DEBUG and INFO logs degrades performance significantly.
Is it common behavior of every loggers?

Is it advisable to keep DEBUG/INFO level logs enabled in the Production Environment ?
As a requirement we need to log some critical information into production environment, but not with a cost of performance.

~Please Advice in this situation.

Thanks,
Nikhil

[Jeanne Boyarsky]

Nikhil Joshi wrote:Is it common behavior of every loggers?

Yes. Output has a cost. Under load that cost becomes more obvious.

Nikhil Joshi wrote:
Is it advisable to keep DEBUG/INFO level logs enabled in the Production Environment ?

No. It is recommended to only log error level in production.

You could try logging the information you absolutely need to another logger so you are only getting that. As opposed to all the debug information.

[edited to fix tag]

[Tim Holloway]

The designers of the various logging frameworks have gone to a great deal of trouble to minimize the overhead of logging.

But there's still a cost. Anytime you start doing a lot of I/O, it's going to add overhead, and if the I/O is synchronous, it will become a potential bottleneck. Even non-synchronous I/O will eventually stall processes when the buffers fill up and things have to be put on hold until they're written out.

That's mostly controllable by adjusting log levels to log only what you really need to see at the moment. The loggers do the level check fairly early in the process so as to reduce overhead for non-logged items.

There's another overhead that can be an issue as well. There can be considerable overhead if you do lots of message formatting for your log messages - and a lot of log messages do, since sometimes it's as important to know what's being done as when/whether it's being done. If you actually are going to output the message, that overhead's pretty much the cost of doing business. But when you're talking tracing and debugging, that logging will typically be turned off except when absolutely essential.

To reduce the overhead of formatting log messages that won't be output anyway, make the message formatting conditional. In a typical logging framework, that means something like the following generic code:

if ( log.isLogging(Level.DEBUG) ( {
    String logMessage = "Processing record with key '" + record.getKey() + "'";
   log.debug(logMessage);
}

[Anurag Blore]

One more thing Its correct we should print only the errors in Production. Sometime its needed that we print some parameters/values if an exception occurs this can be printed as log only if exception happens. Another part is you can automate to create a new log file for every day and zip all the log files which are older by a week. This will save the space on server.

[Tim Holloway]

Anurag Blore wrote:One more thing Its correct we should print only the errors in Production. Sometime its needed that we print some parameters/values if an exception occurs this can be printed as log only if exception happens. Another part is you can automate to create a new log file for every day and zip all the log files which are older by a week. This will save the space on server.

These are the things that logging systems shine at, and why they're infinitely preferable to the brute-force System.out.println approach. You can configure logging not only to report at different degrees of verbosity for production, you can switch more verbose levels on simply by modifying the logging config file without changing code. You can route log messages to multiple destinations, each with its own criteria. You can route copies of critical log messages to non-file destinations such as a master log server, email, and/or paging devices.

And you can configure them to rotate logs and compress old logs (and/or ship them off to archive storage).