Prettify json in a streaming log-CodePudding

I am trying to prettify aws logs stream

log format:

2022-09-07T17:14:35.987000 00:00 2022/09/07/[$LATEST]58f517bs4ebcsae033sd953bs734224 {"cold_start":true,"function_memory_size":512,"function_request_id":"a2cfdd5d-4b0b-4cfa-97c2-4af9ba6a1056","level":"ERROR","message":"Unexpected","sampling_rate":1,"timestamp":"2022-09-07T17:14:35.987Z","xray_trace_id":"1-6318d178-0e96c13a1399a74b4f353edd","extra":"{\"error\":\"AccessDeniedException: User: arn:aws:sts::\",\"stack\":\"AccessDeniedException: User: arn:aws:sts::\"}"}

I want to change this to

2022-09-07T17:14:35.987000 00:00 2022/09/07/[$LATEST]58f51004277b4ebcae033d953b734224 {
  "cold_start": true,
  "function_memory_size": 512,
  "function_request_id": "a2cfdd5d-4b0b-4cfa-97c2-4af9ba6a1056",
  "level": "ERROR",
  "message": "Unexpected",
  "sampling_rate": 1,
  "timestamp": "2022-09-07T17:14:35.987Z",
  "xray_trace_id": "1-6318d178-0e96c13a1399a74b4f353edd",
  "extra": '{"error":"AccessDeniedException: User: arn:aws:sts::","stack":"AccessDeniedException: User: arn:aws:sts::"}',
}

I tried this

aws logs tail /aws/lambda/lambda-name | sed -u -e "s/\({.*}\)/$(echo \'\\1\' | jq)/"

but I am getting this error

parse error: Invalid numeric literal at line 2, column 0

How do I do this?

CodePudding user response：

If you feed your log sample to:

jq -rR 'index("{") as $ix | .[0:$ix], ( .[$ix:]|fromjson)'

you will get the log stamp followed by a copy of the corresponding valid JSON.

Your subject line indicates you want to "prettify" the JSON rather than mangle it, so I'll assume you didn't intend the latter.

If you really want the initial '{' to appear with the log stamp, then assuming a bash or bash-like shell, you could pipe the above into:

    sed $'/^[0-9][0-9][0-9][0-9]-/ {N; s/\\n{/{/; }'