Sorry, I am new in coding in Python, I would need to save a json file generated in a for loop as csv for each iteration of the loop.

I wrote a code that works fine to generate the first csv file but then it is overwritten and I did not find a solution yet. Can anyone help me? many thanks

from twarc.client2 import Twarc2
import itertools
import pandas as pd
import csv
import json
import numpy as np

# Your bearer token here
t = Twarc2(bearer_token="AAAAAAAAAAAAAAAAAAAAA....WTW")

# Get a bunch of user handles you want to check:

list_of_names = np.loadtxt("usernames.txt",dtype="str")
# Get the `data` part of every request only, as one list
def get_data(results):
    return list(itertools.chain(*[result['data'] for result in results]))

user_objects = get_data(t.user_lookup(users=list_of_names, usernames=True))

for user in user_objects:

    following = get_data(t.following(user['id']))

    # Do something with the lists
   
    print(f"User: {user['username']} Follows {len(following)} -2")

    json_string = json.dumps(following)
    df = pd.read_json(json_string)
    df.to_csv('output_file.csv')

CodePudding user response：

You need to add a sequence number or some other unique identifier to the filename. The most clear example would be to keep track of a counter, or use a GUID. In the example below I've used a counter that is initialized before your loop, and is incremented in each iteration. This will produce a list of files like output_file_1.csv, output_file_2.csv, output_file_3.csv and so on.

counter = 0

for user in user_objects:
    following = get_data(t.following(user['id']))
    # Do something with the lists
    print(f"User: {user['username']} Follows {len(following)} -2")
    json_string = json.dumps(following)
    df = pd.read_json(json_string)
    df.to_csv('output_file_'   str(counter)   '.csv')
    counter  = 1

We convert the integer to a string, and paste it inbetween the name of your file and its extension.

CodePudding user response：

from twarc.client2 import Twarc2
import itertools
import pandas as pd
import csv
import json
import numpy as np

# Your bearer token here
t = Twarc2(bearer_token="AAAAAAAAAAAAAAAAAAAAA....WTW")

# Get a bunch of user handles you want to check:

list_of_names = np.loadtxt("usernames.txt",dtype="str")
# Get the `data` part of every request only, as one list
def get_data(results):
    return list(itertools.chain(*[result['data'] for result in results]))

user_objects = get_data(t.user_lookup(users=list_of_names, usernames=True))

for idx, user in enumerate(user_objects):

    following = get_data(t.following(user['id']))

    # Do something with the lists
   
    print(f"User: {user['username']} Follows {len(following)} -2")

    json_string = json.dumps(following)
    df = pd.read_json(json_string)
    df.to_csv(f'output_file{str(idx)}.csv')