Converting a huge file with a JSON document per line into a single JSON list-CodePudding

As you can see below I have a string and I want to convert it into standard JSON.

{"domain":"345","path":"/"}
{"domain":"5432","path":"/"}
{"domain":"345","path":"/"}
{"domain":"345","path":"/"}
{"domain":"23456","path":"/"}
{"domain":"2345","path":"/"}
{"domain":"3456","path":"/"}
{"domain":"123","path":"/"}

I have a file of 12GB which contains data exactly like this.

I want to create a new file where data is like this -

[
   {"domain":"345","path":"/"},
   {"domain":"5432","path":"/"},
   {"domain":"345","path":"/"},
   {"domain":"345","path":"/"},
   {"domain":"23456","path":"/"},
   {"domain":"2345","path":"/"},
   {"domain":"3456","path":"/"},
   {"domain":"123","path":"/"}
]

CodePudding user response：

This is one of those rare special cases where it's safe to treat JSON as raw text.

with open('out.json', 'w') as outfile, open('in.jsonl', 'r') as infile:
  outfile.write('[\n')
  for line in infile:
    outfile.write(f'  {line.rstrip()},\n')
  outfile.write(']\n')

CodePudding user response：

Adding to Charles Duffy's answer, you may treat JSON as simple text; and here's a reasonable one-liner:

awk 'BEGIN{print"["}{print "  "$0","}END{print"]"}' infile.json > outfile.json

CodePudding user response：

You can use a Unix command

$ cat Clients.txt
Slide clipboard items to delete them
$ sed -z 's/\n/,/g;s/,$/\n/' Clients.txt