To merge more than one list of table data and save as csv format using pandas-CodePudding

From the following code, When I iterate and print then got all table data but but when store as csv format using pandas then got only first list of table data. How to store all of them into a single csv file. Thanks in advanced.

import requests
import pandas as pd
isins=['LU0526609390:EUR','IE00BHBX0Z19:EUR']

for isin in isins:
    html = requests.get(f'https://markets.ft.com/data/funds/tearsheet/historical?s={isin}').content
    df_list = pd.read_html(html)
    dfs = df_list
    #print(dfs)
    for df in dfs:
        df.to_csv('data.csv' , header=False, index=True)
         #print(df)

CodePudding user response：

Loop was overwited file. This not saves it as one file bur each file for each iteration, but you get what was wrong

isins=['LU0526609390:EUR','IE00BHBX0Z19:EUR']
k = 0
for isin in isins:
    html = requests.get(f'https://markets.ft.com/data/funds/tearsheet/historical?s={isin}').content
    df_list = pd.read_html(html)
    dfs = df_list
    #print(dfs)
    for df in dfs:
        df.to_csv(str(k) 'data.csv' , header=False, index=True)
        #print(df)
        k = k 1

CodePudding user response：

The idea is to collect the data frames in dfs, loop over it and generate csv files.

import requests
import pandas as pd


isins=['LU0526609390:EUR','IE00BHBX0Z19:EUR']


dfs = []
for isin in isins:
    html = requests.get(f'https://markets.ft.com/data/funds/tearsheet/historical?s={isin}').content
    dfs.extend(pd.read_html(html))

df = pd.concat(dfs) 
df.to_csv('data.csv' , header=False, index=True)

CodePudding user response：

an easy answer would be to use pd.concat() to create a new df and save this. What do you want the csv to look like though, as the result of this concatenation would be. ["CSV"]: https://i.stack.imgur.com/BvZ1X.png

I don't know whether this is sufficient, as the data is not really labelled (might be problem, if you plan to search for more than two funds).

import requests
    import pandas as pd
    funds = ['LU0526609390:EUR', 'IE00BHBX0Z19:EUR']

    for fund in funds:
        html = requests.get(f'https://markets.ft.com/data/funds/tearsheet/historical?s={fund}').content
        df_list = pd.read_html(html)
        df_final = pd.concat(df_list)
        # print(df_final)
        df_final.to_csv('data.csv', header=False, index=True)

(I replaced isin with fund, as isin is already used in python.)