python define function that retrieves data from API and then put into dataframe-CodePudding

I need to get data from an API and then convert into a pandas dataframe. So far I am able to achieve that using for loops (that was efficient when I had one dataset). Now that I have several dataframes, I was wondering how can i write a function that extracts the data from the API and builds a dataframe.

Sample code:

motion_fourth = ['sensor1_id', 'sensor2_id', 'sensor3_id']
motion_fifth = ['sensor4_id', 'sensor5_id', 'sensor6_id']
motion_results_fourth = []
motion_results_fifth = []
        
        # Iterate get urls and retrieve json files
        for sensor in motion_sensors_fourth:
            res = requests.get(f'https://apiv2.XXXX/sensors/{sensor}/Event/history?tstart={timestamp1}&tend={timestamp2}', headers=headers)
            if res.status_code == 200:
                motion_results_fourth.append(res.json())
            else:
                print(f'Request to {sensor} failed.')
                


# Create motion dataframes
 sensor1_motion = pd.DataFrame(motion_results_fourth[0])
 sensor2_motion = pd.DataFrame(motion_results_fourth[1])
 sensor3_motion = pd.DataFrame(motion_results_fourth[2])

Then after completing this for loop, and converting to dataframe I would need to repeat it again for the motion_fifth... So my question is how can I define a function that retrieves the API data, puts into dataframe for several lists of sensor IDs (aka motion_fourth, motion_fifth, etc.)

CodePudding user response：

I suggest that you create a function that receives a generic list as argument. So you can run the same function for both fourth and fifth.

Somewhat like this:

def create_motion_df(motion_sensor_list)
    # Iterate get urls and retrieve json files
    for sensor in motion_sensor_list:
        res = requests.get(f'https://apiv2.XXXX/sensors/{sensor}/Event/history?tstart={timestamp1}&tend={timestamp2}', headers=headers)
        if res.status_code == 200:
            motion_sensor_list.append(res.json())
        else:
            print(f'Request to {sensor} failed.')

    sensor1_motion = pd.DataFrame(motion_sensor_list[0])
    sensor2_motion = pd.DataFrame(motion_sensor_list[1])
    sensor3_motion = pd.DataFrame(motion_sensor_list[2])
    
    return sensor1_motion, sensor2_motion, sensor3_motion

CodePudding user response：

Maybe you can try storing the data in dictionaries.

In my example below, I created one dictionary for a list of sensors per motion. And one dictionary where we can store a dictionary per motion. In that nested dictionary you link the results to the sensors.

I haven't tested it, but maybe you can try it out.

motions = {'fourth':['sensor1_id', 'sensor2_id', 'sensor3_id'],
'fifth' = ['sensor4_id', 'sensor5_id', 'sensor6_id']}

motion_results = {'fourth':{},'fifth':{}}

for motion, sensors in motions.values():
    for sensor in sensors:
        res = requests.get(f'https://apiv2.XXXX/sensors/{sensor}/Event/history?tstart={timestamp1}&tend={timestamp2}', headers=headers)
        if res.status_code == 200:
            motion_results[motion][sensor] = res.json()
        else:
            print(f'Request to {motion} - {sensor} failed.')