• Post Reply Bookmark Topic Watch Topic
  • New Topic

unix shell script - search string and formatting to json  RSS feed

 
Baskar Sikkayan
Ranch Hand
Posts: 32
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have a log file with multiple json message that is not formatted.
Trying to write a shell script to search for a string in the logs and format the matching json and write to a file.

For Example :- in the test.log file,

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService -
{"header":{"time":"2015 06:06:13 PM EDT","reqId":"1","user":"baskar"}}

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService -
{"header":{"time":"2015 06:06:13 PM EDT","reqId":"2","user":"raja"}}

type_logfile 2015-04-14 18:06:15,284 INFO ResponseLogService -
{"header":{"time":"2015 06:06:13 PM EDT","reqId":"3","user":"baskar"}}

I use the below command,

grep -i "baskar" test.log | grep -o "{\"header\".*" | python -m json.tool > ~/search_result.log


The above command works fine if the search returns only one json message. If there are multiple matches, this doesnt work and throw the following error.

Unable to decode the json message.

Whereas, if i redirect the matched output to some file and then read the file line by line to format the json works.

grep -i "baskar" test.log | grep -o "{\"header\".*" > ~/search_result.log

while read line
do
name=$line
echo $name | python -m json.tool >> ~/formatted_search_result.log
done < ~/search_result.log


Since, writing to a temp file and then reading from the temp file to format the json hits the performance, looking for some efficient way of doing this.

The expected output is,
When i search for

grep -i "baskar" test.log
,





Thanks,
Baskar.S
 
Tim Holloway
Saloon Keeper
Posts: 18795
74
Android Eclipse IDE Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Have you looked at this? http://www.skorks.com/2013/04/the-best-way-to-pretty-print-json-on-the-command-line/
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!