Home > Software engineering >  I have a list of data frames but when I concat them the structure breaks
I have a list of data frames but when I concat them the structure breaks

Time：03-21

I read a bunch of csv files and place them in a list, but when I concat the header is dublicated and a comma is added. This is my code:

for f in csv_files:
      
    # read the csv file
    df = pd.read_csv(f)
    df_list.append(df)

#add data frames to a list 
RLI_combined = pd.concat(df_list, axis=0)

This is the csv output I have removed a few rows because the files where too large:

entity;business_line_group;conso_level_entity;report_name;line_item;column_item;z_axis;value_text;amount;approval_text ,entity;business_line_group;conso_level_entity;report_name;line_item;column_item;z_axis;value_text;amount;approval_text
456;test;456;C_72_00_a;0050;0010;UNDEFINED;Value 1;40409261.0100539;44630 ,
456;test;456;C_74_00_a;0040;0010;UNDEFINED;Value 1;46860662.1948734;44631 ,
456;test;456;C_74_00_a;0060;0010;UNDEFINED;Value 1;1783648.53838003;44631 ,
456;test;456;C_74_00_a;0070;0010;UNDEFINED;Value 2;7847645.76582712;44631 ,
456;test;456;C_73_00_a;0310;0010;UNDEFINED;Value 2;48100909.2077918;44631 ,
456;test;456;C_74_00_a;0201;0010;UNDEFINED;Value 1;45652287.0078367;44631 ,
456;test;456;C_72_00_a;0590;0010;UNDEFINED;Value 1;19988230.281333;44630 ,
456;test;456;C_73_00_a;0480;0010;UNDEFINED;Value 1;28243908.6235795;44630 ,
456;test;456;C_73_00_a;0490;0010;UNDEFINED;Value 1;12655653.8647408;44630 ,
456;test;456;C_72_00_a;0450;0030;UNDEFINED;Applicable weight;0.75;Applicable weight ,
456;test;456;C_72_00_a;0460;0030;UNDEFINED;Applicable weight;0.75;Applicable weight ,
456;test;456;C_73_00_a;0040;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0070;0050;UNDEFINED;Applicable weight;0.15;Applicable weight ,
456;test;456;C_73_00_a;0090;0050;UNDEFINED;Applicable weight;0.03;Applicable weight ,
456;test;456;C_73_00_a;0110;0050;UNDEFINED;Applicable weight;0.10;Applicable weight ,
456;test;456;C_73_00_a;0260;0050;UNDEFINED;Applicable weight;0.40;Applicable weight ,
456;test;456;C_73_00_a;0310;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0480;0050;UNDEFINED;Applicable weight;0.05;Applicable weight ,
456;test;456;C_73_00_a;0490;0050;UNDEFINED;Applicable weight;0.10;Applicable weight ,
456;test;456;C_73_00_a;0530;0050;UNDEFINED;Applicable weight;0.40;Applicable weight ,
456;test;456;C_73_00_a;0570;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0590;0050;UNDEFINED;Applicable weight;0.05;Applicable weight ,
456;test;456;C_73_00_a;0080;0050;UNDEFINED;Applicable weight;0.05;Applicable weight ,
456;test;456;C_73_00_a;0140;0050;UNDEFINED;Applicable weight;0.05;Applicable weight ,
456;test;456;C_73_00_a;0150;0050;UNDEFINED;Applicable weight;0.25;Applicable weight ,
456;test;456;C_73_00_a;0170;0050;UNDEFINED;Applicable weight;0.25;Applicable weight ,
456;test;456;C_73_00_a;0190;0050;UNDEFINED;Applicable weight;0.25;Applicable weight ,
456;test;456;C_73_00_a;0200;0050;UNDEFINED;Applicable weight;0.25;Applicable weight ,
456;test;456;C_73_00_a;0250;0050;UNDEFINED;Applicable weight;0.20;Applicable weight ,
456;test;456;C_73_00_a;0280;0050;UNDEFINED;Applicable weight;0.20;Applicable weight ,
456;test;456;C_73_00_a;0290;0050;UNDEFINED;Applicable weight;0.10;Applicable weight ,
456;test;456;C_73_00_a;0360;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;0370;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0380;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0390;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0400;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0420;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0430;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0450;0050;UNDEFINED;Applicable weight;0.50;Applicable weight ,
456;test;456;C_73_00_a;0035;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;0180;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0204;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0206;0050;UNDEFINED;Applicable weight;0.20;Applicable weight ,
456;test;456;C_73_00_a;0207;0050;UNDEFINED;Applicable weight;0.40;Applicable weight ,
456;test;456;C_73_00_a;0220;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0230;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0300;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0510;0050;UNDEFINED;Applicable weight;0.05;Applicable weight ,
456;test;456;C_73_00_a;0520;0050;UNDEFINED;Applicable weight;0.10;Applicable weight ,
456;test;456;C_73_00_a;0540;0050;UNDEFINED;Applicable weight;0.40;Applicable weight ,
456;test;456;C_73_00_a;0560;0050;UNDEFINED;Applicable weight;0.75;Applicable weight ,
456;test;456;C_73_00_a;0600;0050;UNDEFINED;Applicable weight;0.30;Applicable weight ,
456;test;456;C_73_00_a;0610;0050;UNDEFINED;Applicable weight;0.40;Applicable weight ,
456;test;456;C_73_00_a;0630;0050;UNDEFINED;Applicable weight;0.10;Applicable weight ,
456;test;456;C_73_00_a;0640;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0660;0050;UNDEFINED;Applicable weight;0.05;Applicable weight ,
456;test;456;C_73_00_a;0670;0050;UNDEFINED;Applicable weight;0.30;Applicable weight ,
456;test;456;C_73_00_a;0680;0050;UNDEFINED;Applicable weight;0.40;Applicable weight ,
456;test;456;C_73_00_a;0700;0050;UNDEFINED;Applicable weight;0.75;Applicable weight ,
456;test;456;C_73_00_a;0710;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0890;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;0900;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0913;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0914;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0915;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0916;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0917;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0918;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_73_00_a;0940;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;0950;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;0960;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;0970;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;0980;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;0990;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;1000;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;1010;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;1030;0050;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_73_00_a;1040;0050;UNDEFINED;Applicable weight;0.07;Applicable weight ,
456;test;456;C_73_00_a;1050;0050;UNDEFINED;Applicable weight;0.15;Applicable weight ,
456;test;456;C_73_00_a;1060;0050;UNDEFINED;Applicable weight;0.25;Applicable weight ,
456;test;456;C_73_00_a;1070;0050;UNDEFINED;Applicable weight;0.30;Applicable weight ,
456;test;456;C_73_00_a;1080;0050;UNDEFINED;Applicable weight;0.35;Applicable weight ,
456;test;456;C_73_00_a;1090;0050;UNDEFINED;Applicable weight;0.50;Applicable weight ,
456;test;456;C_73_00_a;1100;0050;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0040;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0060;0080;UNDEFINED;Applicable weight;0.50;Applicable weight ,
456;test;456;C_74_00_a;0070;0080;UNDEFINED;Applicable weight;0.50;Applicable weight ,
456;test;456;C_74_00_a;0090;0080;UNDEFINED;Applicable weight;0.50;Applicable weight ,
456;test;456;C_74_00_a;0201;0080;UNDEFINED;Applicable weight;0.20;Applicable weight ,
456;test;456;C_74_00_a;0260;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0080;0080;UNDEFINED;Applicable weight;0.50;Applicable weight ,
456;test;456;C_74_00_a;0130;0080;UNDEFINED;Applicable weight;0.05;Applicable weight ,
456;test;456;C_74_00_a;0150;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0170;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0190;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0180;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0230;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0160;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0210;0080;UNDEFINED;Applicable weight;1.00;Applicable weight ,
456;test;456;C_74_00_a;0269;0080;UNDEFINED;Applicable weight;0.00;Applicable weight ,
456;test;456;C_74_00_a;0273;0080;UNDEFINED;Applicable weight;0.07;Applicable weight ,
456;test;456;C_74_00_a;0277;0080;UNDEFINED;Applicable weight;0.15;Applicable weight ,
,456;test;456;C_73_00_a;0200;0050;UNDEFINED;Applicable weight;0.25;Applicable weight
,456;test;456;C_73_00_a;0250;0050;UNDEFINED;Applicable weight;0.20;Applicable weight
,456;test;456;C_73_00_a;0280;0050;UNDEFINED;Applicable weight;0.20;Applicable weight
,456;test;456;C_73_00_a;0290;0050;UNDEFINED;Applicable weight;0.10;Applicable weight
,456;test;456;C_73_00_a;0360;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;0370;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0380;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0390;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0400;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0420;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0430;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0450;0050;UNDEFINED;Applicable weight;0.50;Applicable weight
,456;test;456;C_73_00_a;0035;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;0180;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0204;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0206;0050;UNDEFINED;Applicable weight;0.20;Applicable weight
,456;test;456;C_73_00_a;0207;0050;UNDEFINED;Applicable weight;0.40;Applicable weight
,456;test;456;C_73_00_a;0220;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0230;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0300;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0510;0050;UNDEFINED;Applicable weight;0.05;Applicable weight
,456;test;456;C_73_00_a;0520;0050;UNDEFINED;Applicable weight;0.10;Applicable weight
,456;test;456;C_73_00_a;0540;0050;UNDEFINED;Applicable weight;0.40;Applicable weight
,456;test;456;C_73_00_a;0560;0050;UNDEFINED;Applicable weight;0.75;Applicable weight
,456;test;456;C_73_00_a;0600;0050;UNDEFINED;Applicable weight;0.30;Applicable weight
,456;test;456;C_73_00_a;0610;0050;UNDEFINED;Applicable weight;0.40;Applicable weight
,456;test;456;C_73_00_a;0630;0050;UNDEFINED;Applicable weight;0.10;Applicable weight
,456;test;456;C_73_00_a;0640;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0660;0050;UNDEFINED;Applicable weight;0.05;Applicable weight
,456;test;456;C_73_00_a;0670;0050;UNDEFINED;Applicable weight;0.30;Applicable weight
,456;test;456;C_73_00_a;0680;0050;UNDEFINED;Applicable weight;0.40;Applicable weight
,456;test;456;C_73_00_a;0700;0050;UNDEFINED;Applicable weight;0.75;Applicable weight
,456;test;456;C_73_00_a;0710;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0890;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;0900;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0913;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0914;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0915;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0916;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0917;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0918;0050;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_73_00_a;0940;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;0950;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;0960;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;0970;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;0980;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;0990;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_73_00_a;1000;0050;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_74_00_a;0180;0080;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_74_00_a;0230;0080;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_74_00_a;0160;0080;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_74_00_a;0210;0080;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_74_00_a;0269;0080;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_74_00_a;0273;0080;UNDEFINED;Applicable weight;0.07;Applicable weight
,456;test;456;C_74_00_a;0277;0080;UNDEFINED;Applicable weight;0.15;Applicable weight
,456;test;456;C_74_00_a;0281;0080;UNDEFINED;Applicable weight;0.25;Applicable weight
,456;test;456;C_74_00_a;0285;0080;UNDEFINED;Applicable weight;0.30;Applicable weight
,456;test;456;C_74_00_a;0289;0080;UNDEFINED;Applicable weight;0.35;Applicable weight
,456;test;456;C_74_00_a;0293;0080;UNDEFINED;Applicable weight;0.50;Applicable weight
,456;test;456;C_74_00_a;0301;0080;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_74_00_a;0303;0080;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_74_00_a;0309;0080;UNDEFINED;Applicable weight;0.00;Applicable weight
,456;test;456;C_74_00_a;0313;0080;UNDEFINED;Applicable weight;0.07;Applicable weight
,456;test;456;C_74_00_a;0317;0080;UNDEFINED;Applicable weight;0.15;Applicable weight
,456;test;456;C_74_00_a;0321;0080;UNDEFINED;Applicable weight;0.25;Applicable weight
,456;test;456;C_74_00_a;0325;0080;UNDEFINED;Applicable weight;0.30;Applicable weight
,456;test;456;C_74_00_a;0329;0080;UNDEFINED;Applicable weight;0.35;Applicable weight
,456;test;456;C_74_00_a;0333;0080;UNDEFINED;Applicable weight;0.50;Applicable weight
,456;test;456;C_74_00_a;0341;0080;UNDEFINED;Applicable weight;0.50;Applicable weight
,456;test;456;C_74_00_a;0343;0080;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_74_00_a;0345;0080;UNDEFINED;Applicable weight;1.00;Applicable weight
,456;test;456;C_72_00_a;0070;0010;UNDEFINED;Value 1;05198630.14;28-feb-22
,456;test;456;C_72_00_a;0190;0010;UNDEFINED;Value 1;835892217;28-feb-22
,456;test;456;C_72_00_a;0260;0010;UNDEFINED;Value 1;4745984333;28-feb-22
,456;test;456;C_73_00_a;0035;0010;UNDEFINED;Value 2;25424822307.28;28-feb-22
,456;test;456;C_73_00_a;0070;0010;UNDEFINED;Value 2;-33216232069.67;28-feb-22
,456;test;456;C_73_00_a;0080;0010;UNDEFINED;Value 1;-20966122130.53;28-feb-22
,456;test;456;C_73_00_a;0110;0010;UNDEFINED;Value 1;-9384698955.80;28-feb-22
,456;test;456;C_73_00_a;0230;0010;UNDEFINED;Value 1;2193605666.84;28-feb-22
,456;test;456;C_73_00_a;0250;0010;UNDEFINED;Value 1;-573769151.28;28-feb-22
,456;test;456;C_73_00_a;0260;0010;UNDEFINED;Value 1;3333715453.55;28-feb-22
,456;test;456;C_73_00_a;0918;0010;UNDEFINED;Value 1;124366;28-feb-22
,456;test;456;C_74_00_a;0160;0010;UNDEFINED;Value 5;-54345799619.07;28-feb-22
,456;test;456;C_74_00_a;0260;0010;UNDEFINED;Value 5;150348.16;28-feb-22
,456;test;456;C_73_00_a;1100;0010;UNDEFINED;Value 5;-37633449687.15;28-feb-22
,456;test;456;C_73_00_a;1100;0020;UNDEFINED;Value 5;-3764349687.15;28-feb-22
,456;test;456;C_73_00_a;1040;0040;UNDEFINED;Value 3;33764349687.15;28-feb-22
,456;test;456;C_73_00_a;1045;0040;UNDEFINED;Value 3;33764349687.15;28-feb-22
,456;test;456;C_73_00_a;1045;0030;UNDEFINED;Value 3;335098209.05;28-feb-22
,456;test;456;C_73_00_a;1040;0010;UNDEFINED;Value 3;7449687.15;28-feb-22
,456;test;456;C_73_00_a;1045;0010;UNDEFINED;Value 1;76449687.15;28-feb-22

Here is two of the files I try to concat:

CSV1:

entity;business_line_group;conso_level_entity;report_name;line_item;column_item;z_axis;value_text;amount;approval_text
456;test;456;C_72_00_a;0070;0010;UNDEFINED;Value 1;05198630.14;28-feb-22
456;test;456;C_72_00_a;0190;0010;UNDEFINED;Value 1;835892217;28-feb-22
456;test;456;C_72_00_a;0260;0010;UNDEFINED;Value 1;4745984333;28-feb-22
456;test;456;C_73_00_a;0035;0010;UNDEFINED;Value 2;25424822307.28;28-feb-22
456;test;456;C_73_00_a;0070;0010;UNDEFINED;Value 2;-33216232069.67;28-feb-22
456;test;456;C_73_00_a;0080;0010;UNDEFINED;Value 1;-20966122130.53;28-feb-22
456;test;456;C_73_00_a;0110;0010;UNDEFINED;Value 1;-9384698955.80;28-feb-22
456;test;456;C_73_00_a;0230;0010;UNDEFINED;Value 1;2193605666.84;28-feb-22
456;test;456;C_73_00_a;0250;0010;UNDEFINED;Value 1;-573769151.28;28-feb-22
456;test;456;C_73_00_a;0260;0010;UNDEFINED;Value 1;3333715453.55;28-feb-22
456;test;456;C_73_00_a;0918;0010;UNDEFINED;Value 1;124366;28-feb-22
456;test;456;C_74_00_a;0160;0010;UNDEFINED;Value 5;-54345799619.07;28-feb-22
456;test;456;C_74_00_a;0260;0010;UNDEFINED;Value 5;150348.16;28-feb-22
456;test;456;C_73_00_a;1100;0010;UNDEFINED;Value 5;-37633449687.15;28-feb-22
456;test;456;C_73_00_a;1100;0020;UNDEFINED;Value 5;-3764349687.15;28-feb-22
456;test;456;C_73_00_a;1040;0040;UNDEFINED;Value 3;33764349687.15;28-feb-22
456;test;456;C_73_00_a;1045;0040;UNDEFINED;Value 3;33764349687.15;28-feb-22
456;test;456;C_73_00_a;1045;0030;UNDEFINED;Value 3;335098209.05;28-feb-22
456;test;456;C_73_00_a;1040;0010;UNDEFINED;Value 3;7449687.15;28-feb-22
456;test;456;C_73_00_a;1045;0010;UNDEFINED;Value 1;76449687.15;28-feb-22

CSV 2:

entity;business_line_group;conso_level_entity;report_name;line_item;column_item;z_axis;value_text;amount;approval_text 
456;test;456;C_72_00_a;0050;0010;UNDEFINED;Value 1;40409261.0100539;44630 
456;test;456;C_74_00_a;0040;0010;UNDEFINED;Value 1;46860662.1948734;44631 
456;test;456;C_74_00_a;0060;0010;UNDEFINED;Value 1;1783648.53838003;44631 
456;test;456;C_74_00_a;0070;0010;UNDEFINED;Value 2;7847645.76582712;44631 
456;test;456;C_73_00_a;0310;0010;UNDEFINED;Value 2;48100909.2077918;44631 
456;test;456;C_74_00_a;0201;0010;UNDEFINED;Value 1;45652287.0078367;44631 
456;test;456;C_72_00_a;0590;0010;UNDEFINED;Value 1;19988230.281333;44630 
456;test;456;C_73_00_a;0480;0010;UNDEFINED;Value 1;28243908.6235795;44630 
456;test;456;C_73_00_a;0490;0010;UNDEFINED;Value 1;12655653.8647408;44630 
456;test;456;C_73_00_a;0530;0010;UNDEFINED;Value 1;27792100.4510517;44630 
456;test;456;C_73_00_a;0570;0010;UNDEFINED;Value 1;20768476.5051213;44630 
456;test;456;C_73_00_a;0480;0010;UNDEFINED;Value 5;28601515.4535418;44631 
456;test;456;C_73_00_a;0490;0010;UNDEFINED;Value 5;17269663.9202129;44631 
456;test;456;C_73_00_a;0530;0010;UNDEFINED;Value 5;21250486.2477187;44631 
456;test;456;C_73_00_a;0570;0010;UNDEFINED;Value 5;12924566.8399212;44631 
456;test;456;C_73_00_a;0110;0010;UNDEFINED;Value 3;17299383.641137;44630 
456;test;456;C_73_00_a;0035;0010;UNDEFINED;Value 3;19054145.8837998;44630 
456;test;456;C_72_00_a;0280;0010;UNDEFINED;Value 3;294348.91379545;44630 
456;test;456;C_73_00_a;0340;0010;UNDEFINED;Value 3;40803729.9712868;44630 
456;test;456;C_74_00_a;0240;0010;UNDEFINED;Value 1;25387904.3875074;44630 
456;test;456;C_73_00_a;0340;0010;UNDEFINED;Value 5;6951075.43742419;44630 
456;test;456;C_74_00_a;0240;0010;UNDEFINED;Value 3;12298844.1430509;44630 
456;test;456;C_72_00_a;0040;0030;UNDEFINED;Applicable weight;1.00;Applicable weight 
456;test;456;C_72_00_a;0050;0030;UNDEFINED;Applicable weight;1.00;Applicable weight 
456;test;456;C_72_00_a;0060;0030;UNDEFINED;Applicable weight;1.00;Applicable weight 
456;test;456;C_72_00_a;0070;0030;UNDEFINED;Applicable weight;1.00;Applicable weight 
456;test;456;C_72_00_a;0070;0010;UNDEFINED;Value 1;05198630.14;28-feb-22 
456;test;456;C_72_00_a;0190;0010;UNDEFINED;Value 1;835892217;28-feb-22 
456;test;456;C_72_00_a;0260;0010;UNDEFINED;Value 1;4745984333;28-feb-22 
456;test;456;C_73_00_a;0035;0010;UNDEFINED;Value 2;25424822307.28;28-feb-22 
456;test;456;C_73_00_a;0070;0010;UNDEFINED;Value 2;-33216232069.67;28-feb-22 
456;test;456;C_73_00_a;0080;0010;UNDEFINED;Value 1;-20966122130.53;28-feb-22 
456;test;456;C_73_00_a;0110;0010;UNDEFINED;Value 1;-9384698955.80;28-feb-22 
456;test;456;C_73_00_a;0230;0010;UNDEFINED;Value 1;2193605666.84;28-feb-22 
456;test;456;C_73_00_a;0250;0010;UNDEFINED;Value 1;-573769151.28;28-feb-22 
456;test;456;C_73_00_a;0260;0010;UNDEFINED;Value 1;3333715453.55;28-feb-22 
456;test;456;C_73_00_a;0918;0010;UNDEFINED;Value 1;124366;28-feb-22 
456;test;456;C_74_00_a;0160;0010;UNDEFINED;Value 5;-54345799619.07;28-feb-22 
456;test;456;C_74_00_a;0260;0010;UNDEFINED;Value 5;150348.16;28-feb-22 
456;test;456;C_73_00_a;1100;0010;UNDEFINED;Value 5;-37633449687.15;28-feb-22 
456;test;456;C_73_00_a;1100;0020;UNDEFINED;Value 5;-3764349687.15;28-feb-22 
456;test;456;C_73_00_a;1040;0040;UNDEFINED;Value 3;33764349687.15;28-feb-22 
456;test;456;C_73_00_a;1045;0040;UNDEFINED;Value 3;33764349687.15;28-feb-22 
456;test;456;C_73_00_a;1045;0030;UNDEFINED;Value 3;335098209.05;28-feb-22 
456;test;456;C_73_00_a;1040;0010;UNDEFINED;Value 3;7449687.15;28-feb-22 
456;test;456;C_73_00_a;1045;0010;UNDEFINED;Value 1;76449687.15;28-feb-22

I hope you can point me in the right direction

CodePudding user response：

Pandas read.csv have sep=',' as a default, which is not your case as your values are separated by ';'. Try:

pd.read_csv(f,sep=';')

As a general advise, ensure you are reading the dataframes properly before trying other operations.

CodePudding user response：

I think you need to specify the separator character. Your file uses semicolumn ; which is not the standard comma ,. Try using df = pd.read_csv(f, sep=";")

CodePudding user response：

Try:

df = pd.read_csv(csv_files[0],delimiter=";")

for f in csv_files[1:]:
  df = pd.concat([df,pd.read_csv(f,delimiter=";")],ignore_index=True)

Page link：https//www.codepudding.com/Softwareengineering/343152.html

Prev:Pandas select a specific column from each row
Next:Remove string after a predefined string from a nested list of dictionaries with pandas
  • Related

About Us:  Contact Us      Terms of Service       Privacy Policy

Copyright © 2010-2023，Powered By CodePudding