I'm trying to get the specific values from dict and use in the requests module to make the get requests.

clusters = {
    'cluster_1':'https://cluster_1.something.com/api',
    'cluster_2':'https://cluster_2.something.com/api'
}

Code:

def getCsv():
    for i in clusters:
        r = requests.get(i.values(), headers=headers, params=params)
        with open("input.csv", "w") as f:
            f.writelines(r.text.splitlines(True))
        df = pd.read_csv("input.csv")
        return df

getCsv()

Am I doing it right?

Also final step is to print the clusters key into output csv using below code.

with open(rb'output.csv', 'w', newline='') as out_file:
        timestamp = datetime.now()        
        df = getCsv()
        if 'Name' in df.columns:
            df.rename(columns = {"Name": "team", "Total": "cost"}, inplace = True)
        df.insert(0, 'date',timestamp)
        df.insert(1, 'resource_type', "pod")
        df.insert(2, 'resource_name', "kubernetes")
        df.insert(3, 'cluster_name', i.keys)
        df.drop(["CPU", "GPU", "RAM", "PV", "Network", "LoadBalancer", "External", "Shared", "Efficiency"], axis=1, inplace=True)
        df['team'] = df['team'].map(squads).fillna(df['team'])
        df.groupby(["date","resource_type","resource_name","cluster_name","team"]).agg({"cost": sum}).reset_index().to_csv(out_file, index=False)

But seems not working, any guidance will be appreciated.

CodePudding user response：

The function getCsv is screwed up. You're passing multiple urls to requests.get by passing i.values() and you're returning in the first iteration of the for loop.

Try extracting the loop from the function and reformulating the function like this:

def get_cluster_df(url, **kwargs):
    r = requests.get(url, **kwargs)
    with open("input.csv", "w") as f:
        f.writelines(r.text.splitlines(True))
    return pd.read_csv("input.csv")

headers = ...
params = ...

dfs = {}
for name, url in clusters.items():
    dfs[name] = get_cluster_df(url, headers=headers, params=params)

You will end up with a dictionary dfs which will look like:

{
    'cluster_1': pd.DataFrame(),
    'cluster_2': pd.DataFrame()
}

CodePudding user response：

To get the key/value pairs from a Python dictionary and use them in a GET request using the requests module, you can use the params parameter of the requests.get() function. This parameter takes a dictionary of key/value pairs and encodes them as query parameters in the URL.

Here is an example of how you can do this: import requests

Set up the dictionary of key/value pairs

params = {'key1': 'value1', 'key2': 'value2'}

Make the GET request using the params parameter

response = requests.get('http://httpbin.org/get', params=params)

Print the response text

print(response.text)

This will send a GET request to the URL http://httpbin.org/get?key1=value1&key2=value2, with the key/value pairs encoded as query parameters.

You can also use the json parameter of the requests.get() function to send a JSON object as the request body. This is useful if you want to send a complex data structure as part of the request, rather than encoding it as query parameters in the URL.