Pandas add rows dynamically via API data fetch-CodePudding

I'm trying to pull data from an API and insert that data into a Pandas dataframe.

I'm retrieving all the necessary data fine but my issue is that each row index isn't incrementing and only has a row index of 0 for all results, so when I export the data it only shows 1 row of results.

Here is my code. I have the dataframe wrapped in a for loop where the data is coming from:

import pandas as pd

for item in response['items']:

    df = pd.DataFrame({
        'Title': [item['snippet']['title']],
        'Description': [item['snippet']['description']],
        'Date Posted': [item['snippet']['publishedAt']],
    })

    print(df)

CodePudding user response：

It's bad practice to add on to a dataframe. Try something like this:

data = []

for item in response['items']:
    data.append({
        'Title': item['snippet']['title'],
        'Description': item['snippet']['description'],
        'Date Posted': item['snippet']['publishedAt'],
    })

df = pd.DataFrame(data)
print(df)

CodePudding user response：

Maybe, you can use pd.json_normalize:

cols = ['title', 'description', 'publishedAt']
df = pd.json_normalize(response['items'], 'snippet')[cols]