I'm trying to scrape some information from a website through a JSON file hosted on the cloud.

1. Scrape JSON hosted on a website
2. Find out what's the new key:value pair
3. Append the values to a dictionary
4. Turn the dictionary into a pandas DF

I've been able compare the new JSON file against the old one and determine what's new.

import json
new_file = open('Fruit_new.json')
new_json = json.load(new_file)

old_file = open('Fruits_old.json')
old_json = json.load(old_file)

x = {}

for i in new_json['fruits']:
    if i not in old_json['fruits']:
         print(i)

Output:

{"name": "Grapes","colour": "Purple"}
{"name": "Watermelon","colour": "Green"}

I want to move the output to a pandas DF, but I'm not sure how I'm supposed to go about it. My understanding is I have to somehow turn the output into a dictionary first.

Ultimately, I want the output to look like:

name        colour
Grapes      Purple
Watermelon  Green

Once I have the DF, then it will be much easier to write it on Google Sheets.

CodePudding user response：

Try this:

df = pd.DataFrame(columns = ['name','colour'])
for dict_ in your_dicts:
    df = df.append(dict_, ignore_index=True)