• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Format of the sentences used in the log messages.

 
Ranch Hand
Posts: 157
Netbeans IDE Postgres Database Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello:

I've another doubt about logging. In this case, it's about the format of the sentences used in the log messages. I'm not referring to the information that is included together with the sentence (time stamp, level, etc.), but to the sentence itself.

Usually, people tend to use natural sentences in the messages destined to the INFO level and above. For the DEBUG level and below, they use expressions much more technical. Or even cryptic ones, which only include values, without specifying what each value means.

During my day-to-day work, I've adopted a style that I apply to all levels. It consists in outputting the name of an event, followed by keys and values related to the context in which the logging occurs. In other words, the name of the method where the message is being logged, plus the names and values of certain variables. For example, instead of outputing this:



I output this:



This allows me to process mentally the information much faster, and to automatize easily some tasks in that logs, such as filtering certain type of data. In my opinion, logs are not for typical users, but for users with at least some technical knowledge of the application, such as developers, testers or advanced users. So I don't think that the messages need to be so human-oriented as the ones shown in the interface of the own application, be it a text console or a GUI. What do you opine about all this?

Thank you.
 
Rancher
Posts: 1093
29
Netbeans IDE Oracle MySQL Database Tomcat Server C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I like your format better, and I agree: it is a difference audience that you write for when you do log files--people that read them tend to be more left brain dependent, so more info in a smaller space is better instead of flowery sentences.
 
Sheriff
Posts: 17644
300
Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You can take it a step further and format all or a subset of your log messages so that they are machine readable. That way, it's easier to automate. You could put a marker on the machine-readable log messages or you could append them to a separate log altogether.
 
Avor Nadal
Ranch Hand
Posts: 157
Netbeans IDE Postgres Database Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Les Morgan wrote:I like your format better, and I agree: it is a difference audience that you write for when you do log files--people that read them tend to be more left brain dependent, so more info in a smaller space is better instead of flowery sentences.



Wonderful. I was afraid that the advantages of this format were too subjective. Thanks.

Junilu Lacar wrote:You can take it a step further and format all or a subset of your log messages so that they are machine readable. That way, it's easier to automate. You could put a marker on the machine-readable log messages or you could append them to a separate log altogether.



What do you mean with "machine readable"? I'm not sure if I've understand you correctly. Do you refer to some kind of binary format?
 
Junilu Lacar
Sheriff
Posts: 17644
300
Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Not everything that is machine readable has to be in binary format. JSON, for example, is machine readable as well as human readable. It's machine readable because it has a specific format that can be processed programmatically. CSV is another machine readable format. Properties (key=value pairs) are also machine readable. Free formatted prose does not lend itself well to being machine readable with current technology. You'd need to have some kind of natural language parser and interpreter. Maybe IBM Watson will lead us there someday, but I digress. Anyway, I hope that clarifies what I meant.
 
Sheriff
Posts: 5555
326
IntelliJ IDE Python Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
In your other topic about Logging I mentioned that the company I currently work for uses Splunk to capture the logging output from application servers. I also mentioned that it also serves as a pretty darn good data analysis tool too. Well Splunk loves key=value pairs and is really good at finding them quickly, so taking your example I could write a simple search for "sessionId=97fd45ef21225b079967efd" to obtain an audit of everything that occurred during that user session.

There are other tools that do similar things of course, but having log output that can be easily parsed programmatically gives you lots of scope for doing automated analysis and reporting. Our production support team have specified a particular String that should appear in the log output only when something really bad happens that requires immediate attention. Something like "SEVERE_ERROR_IMMEDIATE_ATTN" that their systems are configured to raise a production incident ticket when observed in log output. These are the types of events that get you a phone call at 3am. It's all about responding to problems quickly and a simple scheme such as this in your log output can be a tremendous help.
 
Avor Nadal
Ranch Hand
Posts: 157
Netbeans IDE Postgres Database Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Junilu Lacar wrote:Not everything that is machine readable has to be in binary format. JSON, for example, is machine readable as well as human readable. It's machine readable because it has a specific format that can be processed programmatically. CSV is another machine readable format. Properties (key=value pairs) are also machine readable. Free formatted prose does not lend itself well to being machine readable with current technology. You'd need to have some kind of natural language parser and interpreter. Maybe IBM Watson will lead us there someday, but I digress. Anyway, I hope that clarifies what I meant.



OK, clarified. So you referred to use a syntax that is standard, not a custom implementation. Right? Before posting my questions here, I read at least 2 articles that used JSON as output, just as you recommend. They called it "structured log". One of that articles was Structured Logs at Nextdoor. I think I'll give it a try. It shouldn't cost much to move from one format to the other one. Thanks.

It's funny, because the application server Wildfly follows an inverse philosophy. Its CLI outputs JSON as result of an operation, whereas its logs are more human-readable than that, he he he.

Tim Cooke wrote:In your other topic about Logging I mentioned that the company I currently work for uses Splunk to capture the logging output from application servers. I also mentioned that it also serves as a pretty darn good data analysis tool too. Well Splunk loves key=value pairs and is really good at finding them quickly, so taking your example I could write a simple search for "sessionId=97fd45ef21225b079967efd" to obtain an audit of everything that occurred during that user session.



Great. Specially the part of generating a ticket automatically. That could save me some time when an error occurs. I took a quick look to the Splunk website some days ago. But I still have to dig in it much more. Again, thank you for the recommendation.
 
reply
    Bookmark Topic Watch Topic
  • New Topic