Saving a nested JSON response of what I'm able to call a print function on?-CodePudding

I'm having a difficult time figuring out how to save the results of an API call to a CSV. When I run the print function below, it prints exactly what I want saved. I've tried dictwriter, I've tried jsondumps, there's always something wrong with the output (putting everything on one row doing columns too, or adding writing like t,h,i,s to the end of each entry).

    fList = sys.argv[1]
with open(fList, "r", encoding='utf-8-sig') as ins:
    for line in ins:
        targetDomain = line
        url1 = "https://websitehere={0}".format(
            targetDomain)
        response = requests.get(url1).json()
        for subs in response["result"]["records"]:
            with open('output.csv', 'a', newline="") as f:
                print(subs)`

The code above prints:

{'domain': '***redacted out of caution***', 'firstSeen': 1629178483, 'lastSeen': 1629178507}
{'domain': '***redacted out of caution***', 'firstSeen': 1631174847, 'lastSeen': 1631174878}
{'domain': '***redacted out of caution***', 'firstSeen': 1630654337, 'lastSeen': 1630654337}
{'domain': '***redacted out of caution***', 'firstSeen': 1630465072, 'lastSeen': 1630465072}

Which is exactly what I want saved in a CSV but cannot for the life of me figure it out. Thank you.

CodePudding user response：

I would use the csv module for this. Example with DictWriter (taken from the docs):

import csv

rows = [
    {'domain': 'xx', 'firstSeen': 1629178483, 'lastSeen': 1629178507},
    {'domain': 'xx', 'firstSeen': 1631174847, 'lastSeen': 1631174878}
]

with open('output.csv', 'w') as out_file:
    writer = csv.DictWriter(out_file, fieldnames=list(rows[0].keys()))

    writer.writeheader()
    for row in rows:
        writer.writerow(row)

CodePudding user response：

I figured it out - Thank you.

with open('output.csv', 'a') as f:
                print(json.dumps(subs), file=f)