I have a data frame that contains a frequency, SNR and time tag. I have a problem in my machine in such way that sometimes the frequency goes back instead of going forward, I've already sorted the data frame according to time but now I want to iterate over the rows and delete the rows in which the frequency is lower than the previous one.

Here is part of my data frame (the entire data frame is around 3700 rows):

        freq         snr    time_tag
128   395.400902  115.737681  652492975
118   395.385596  115.709449  652492976
134   395.415043  115.858103  652492978
137   395.427460  115.805077  652492979
133   395.413911  115.870729  652492980

can you please help? thank you.

CodePudding user response：

Use shift to compare the current row with the previous:

>>> df[~df['freq'].lt(df['freq'].shift())]
           freq         snr   time_tag
128  395.400902  115.737681  652492975
134  395.415043  115.858103  652492978
137  395.427460  115.805077  652492979

Or, suggested by @mozway:

>>> df[df['freq'].ge(df['freq'].shift(fill_value=0))]
           freq         snr   time_tag
128  395.400902  115.737681  652492975
134  395.415043  115.858103  652492978
137  395.427460  115.805077  652492979

CodePudding user response：

You can use an expanding window like this:

increasing_values=df.set_index("time_tag").loc[:,"freq"].expanding().apply(max)
df.loc[df.freq.isin(increasing_values),:]

Output: