Read Linux Path and append all data-CodePudding

I want to read all csv files present in a Linux path and store it in a single data frame using Python.

I am able to read the files but while storing, each file is getting created as dictionary object ex: df['file1'],df['file2'] and so on.

Please let me know how can I store each csv file into separate data frame dynamically and then combine them to store in a single data frame.

Thanks in advance.

CodePudding user response：

from pathlib import Path
import pandas as pd

dataframes = []
for p in Path("path/to/data").iterdir():
    if p.suffix == ".csv":
        dataframes.append(pd.read_csv(p))
        
df = pd.concat(dataframes)

Or if you want to include subdirectories

from pathlib import Path import pandas as pd

path = Path("path/to/data")
df = pd.concat([pd.read_csv(f) for f in path.glob("**/*.csv")], ignore_index=True)