Let's say I have the following pandas.dataframe:

data

      series    time_idx    value
0        0          0      -0.000000
1        0          1       0.018844
2        0          2       0.028694
3        0          3       0.050784
4        0          4       0.067037
...      ...        ...     ...
3995     9          395     0.973978
3996     9          396     0.944002
3997     9          397     1.001089
3998     9          398     1.132001
3999     9          399     1.169244
4000 rows × 3 columns

I want to test if for each series (0..9) the time indexes are incremented by 1 from row to row and if not where the difference is?

I thought about sorting the dataframe by series and by time_index and then compare to the index mod 400, but it's not a nice solution. Any suggestions?

Thanks

CodePudding user response：

The following is based on what I understand from your question. See if this answers your question. I have to use 'True' instead of Boolean True because the dataframe converts it to the numeric 1.0.

df['IncOne'] = (df.series==df.series.shift())
df['IncOne'] = (
    np.where(df.IncOne, 
        np.where( df.time_idx.eq(df.time_idx.shift() 1), 
                  'True' , df.time_idx-df.time_idx.shift() ),
    ''))

CodePudding user response：

Assuming that the dataframe is df you can try this:

df["diff"] = df.groupby(by="series")["time_idx"].diff().fillna(1) != 1

It will create a new column "diff" with boolean values. A True value indicates that the difference between the time_idx value in the current row and the one preceding it is different than one. Only differences between rows corresponding to the same series can give a True value.