Pandas concat fail and appear NaN-CodePudding

I want vertical concat two dataframe but I have no idea why it concact by ['data_label'] and generate NaN?

print(leakage_data1)

    0    1    2   data_label
0  0.2  0.1  0.7      0
1  0.5  0.6  0.4      0


print(noise_data1)

    0    1    2   data_label
0  0.9  0.8  1.0      1
1  0.7  0.5  0.1      1

I use model1_TestData = pd.concat([leakage_data1, noise_data1], axis=0, ignore_index=True)
and got

print(model1_TestData)

    0    1    2   data_label   0    1    2
0  0.2  0.1  0.7      0       NaN  NaN  NaN
1  0.5  0.6  0.4      0       NaN  NaN  NaN
2  NaN  NaN  NaN      1       0.9  0.8  1.0
3  NaN  NaN  NaN      1       0.7  0.5  0.1

How can i fix this problem?

CodePudding user response：

Check the data types on your column names. If one df has a string column name '1' and another is 1, it will print out on the screen the same but will give you the error you see there because to concat you have to have exactly the same column names and 1!='1'

model1_TestData.columns

should reveal something like Index(['0', '1',..., 0, 1], dtype='object')

CodePudding user response：

You need to just use

df.merge(df1, right_index = True, left_index = True)

when you are using axis =0 you are essentially telling it to do it via rows instead of columns...unless you are trying to do that, so you would want your data to simply be more vertical instead of horizontal?

CodePudding user response：

This should work:

df1 = pd.DataFrame({0:[.2, .5], 1:[.1, .6], 2:[.7, .4], 'data_label':0})
df2 = pd.DataFrame({0:[.9, .7], 1:[.8, .5], 2:[1., .1], 'data_label':1})
print(pd.concat([df1, df2]))

Result:

     0    1    2  data_label
0  0.2  0.1  0.7           0
1  0.5  0.6  0.4           0
0  0.9  0.8  1.0           1
1  0.7  0.5  0.1           1