Concat two dataframes: Reindexing only valid with uniquely valued Index objects-CodePudding

I want to concat 2 dataframes vertically, but I get this error:

Reindexing only valid with uniquely valued Index objects

How can I fix it?

df1

      TimeStamp  Input     X     Y  Time  Distance  Speed Pressure Tilt_X Tilt_X
  16    79769.0  touch  8898  8438   NaN       NaN    NaN     None   None   None
  17    79784.0  touch  8898  8438  15.0       0.0    0.0     None   None   None
  18    79793.0  touch  8898  8438   9.0       0.0    0.0     None   None   None
  19    79802.0  touch  8898  8438   9.0       0.0    0.0     None   None   None

df2

      TimeStamp Input     X     Y  Time  Distance  Speed Pressure Tilt_X Tilt_Y
  26    84456.0   pen  8762  9318   NaN       NaN    NaN        0      0      0
  27    84459.0   pen  8762  9318   3.0       0.0    0.0     4069  -1397  -1445
  28    84459.0   pen  8762  9318   0.0       0.0    0.0     4069  -1397  -1445
  29    84464.0   pen  8762  9318   5.0       0.0    0.0     3944  -1397  -1445
  30    84472.0   pen  8762  9318   8.0       0.0    0.0     3692  -1397  -1445
  31    84482.0   pen  8741  9253  10.0       0.6    0.0     3317  -1397  -1445

I want to concat to this:

      TimeStamp  Input     X     Y  Time  Distance  Speed Pressure Tilt_X Tilt_X
  16    79769.0  touch  8898  8438   NaN       NaN    NaN     None   None   None
  17    79784.0  touch  8898  8438  15.0       0.0    0.0     None   None   None
  18    79793.0  touch  8898  8438   9.0       0.0    0.0     None   None   None
  19    79802.0  touch  8898  8438   9.0       0.0    0.0     None   None   None
  26    84456.0    pen  8762  9318   NaN       NaN    NaN        0      0      0
  27    84459.0    pen  8762  9318   3.0       0.0    0.0     4069  -1397  -1445
  28    84459.0    pen  8762  9318   0.0       0.0    0.0     4069  -1397  -1445
  29    84464.0    pen  8762  9318   5.0       0.0    0.0     3944  -1397  -1445
  30    84472.0    pen  8762  9318   8.0       0.0    0.0     3692  -1397  -1445
  31    84482.0    pen  8741  9253  10.0       0.6    0.0     3317  -1397  -1445

But the following code results in Reindexing only valid with uniquely valued Index objects:

input_df = pd.DataFrame(input_lst) 
time_df = pd.DataFrame(time_lst).astype(int)
x_df = pd.DataFrame(x_lst)
y_df = pd.DataFrame(y_lst)
contactid_df = pd.DataFrame(contactid_lst)
pressure_df = pd.DataFrame(pressure_lst)
tiltx_df = pd.DataFrame(tiltx_lst)
tilty_df = pd.DataFrame(tilty_lst)


tmp_df = pd.concat([input_df,time_df,contactid_df,x_df,y_df,pressure_df,tiltx_df,tilty_df],axis = 1)
tmp_df.columns=['Input','Time','ID','X','Y','Pressure','TiltX','TiltY']
touch_df = tmp_df[tmp_df['Input'] == 'Touch']
touch_input_df = touch_df['Input']
touch_Time_df = touch_df['Time']
touch_X_df = touch_df['X']
touch_Y_df = touch_df['Y']
touch_Pressure_df = touch_df['Pressure']
touch_TiltX_df = touch_df['TiltX']
touch_TiltY_df = touch_df['TiltY']

pen_df = tmp_df[tmp_df['Input'] == 'Pen']
pen_input_df = pen_df['Input']
pen_Time_df = pen_df['Time']
pen_X_df = pen_df['X']
pen_Y_df = pen_df['Y']
pen_Pressure_df = pen_df['Pressure']
pen_TiltX_df = pen_df['TiltX']
pen_TiltY_df = pen_df['TiltY']

touch_tmp_df = pd.concat([touch_Time_df,
                          touch_input_df,
                          touch_X_df,
                          touch_Y_df,
                          touch_time_df_diff,
                          touch_xy_real_dis_df,
                          touch_speed_df,
                          touch_Pressure_df,
                          touch_TiltX_df,
                          touch_TiltY_df],axis = 1)    
touch_tmp_df.columns=['TimeStamp',
                      'Input',
                      'X',
                      'Y',
                      'Time',
                      'Distance',
                      'Speed',
                      'Pressure',
                      'Tilt_X',
                      'Tilt_X']

pen_tmp_df = pd.concat([pen_Time_df,
                        pen_input_df,
                        pen_X_df,
                        pen_Y_df,
                        pen_time_df_diff,
                        pen_xy_real_dis_df,
                        pen_speed_df,
                        pen_Pressure_df,
                        pen_TiltX_df,
                        pen_TiltY_df],axis = 1)    
pen_tmp_df.columns=['TimeStamp',
                    'Input',
                    'X',
                    'Y',
                    'Time',
                    'Distance',
                    'Speed',
                    'Pressure',
                    'Tilt_X',
                    'Tilt_Y']


touch = touch_tmp_df[touch_tmp_df['Input'] == 'Touch']
pen = pen_tmp_df[pen_tmp_df['Input'] == 'Pen']

all_df = pd.concat([touch,pen],axis = 0, ignore_index=True)

Error msg:

Traceback (most recent call last):

File "D:\Python\Digiinfo_Parser\Digiinfo_Parser.py", line 276, in 
  <module>
all_df = pd.concat([touch,pen,ptp],axis = 0, ignore_index=True)

File "C:\Users\ANDYCHEN\Anaconda3\lib\site- 
packages\pandas\core\reshape\concat.py", line 298, in concat
return op.get_result()

File "C:\Users\ANDYCHEN\Anaconda3\lib\site- 
packages\pandas\core\reshape\concat.py", line 516, in get_result
indexers[ax] = obj_labels.get_indexer(new_labels)

File "C:\Users\ANDYCHEN\Anaconda3\lib\site- 
packages\pandas\core\indexes\base.py", line 3171, in get_indexer
raise InvalidIndexError(

InvalidIndexError: Reindexing only valid with uniquely valued Index 
objects

CodePudding user response：

Here the InvalidIndexError is actually referring to the column index.

df1 has duplicate column names:

... Pressure Tilt_X Tilt_X

pd.concat does not work with duplicate column names.

In this case it looks like the second Tilt_X should actually be Tilt_Y, but you should check all of your dataframes' columns to make sure there are no other duplicates.

CodePudding user response：

You want to concat vertically, but used axis=1. Remove this parameter:

pd.concat([input_df, time_df, contactid_df, x_df, y_df, pressure_df, tiltx_df, tilty_df])