Add column to DataFrame with threshold for difference between data date in a column and its precedin-CodePudding

Be the following python pandas DataFrame:

| num_ID | start_date  | end_date   | other_column      |
| ------ | ----------- | ---------- | ----------------- |
| 1      | 2022-02-14  | 2022-02-15 | 09:23:00          |
| 1      | 2022-02-20  | 2022-02-25 | 12:10:01          |
| 2      | 2022-03-11  | 2022-03-21 | 08:21:00          |
| 2      | 2022-03-22  | 2022-03-27 | 02:36:00          |
| 2      | 2022-04-10  | 2022-04-15 | 11:43:03          |
| 3      | 2022-02-04  | 2022-02-06 | 16:51:00          |
| 3      | 2022-02-14  | 2022-02-23 | 19:35:10          |
| 4      | 2022-02-28  | 2022-10-12 | 00:01:00          |

For each num_ID value, I want to add a new column total_times with a value 1 if the end_date value of the previous row and the start_date value from the actual one are at least 7 days apart.

First add the value 0 for this column for the first row of each num_ID value, since they do not have a previous row.

| num_ID | start_date  | end_date   | other_column      | total_times |
| ------ | ----------- | ---------- | ----------------- | ----------- |
| 1      | 2022-02-14  | 2022-02-15 | 09:23:00          | 0           |
| 1      | 2022-02-20  | 2022-02-25 | 12:10:01          |             |
| 2      | 2022-03-11  | 2022-03-21 | 08:21:00          | 0           |
| 2      | 2022-03-22  | 2022-03-27 | 02:36:00          |             |
| 2      | 2022-04-10  | 2022-04-15 | 11:43:03          |             |
| 3      | 2022-02-04  | 2022-02-06 | 16:51:00          | 0           |
| 3      | 2022-02-14  | 2022-02-23 | 19:35:10          |             |
| 4      | 2022-02-28  | 2022-10-12 | 00:01:00          | 0           |

Finally we fill in the remaining rows with the condition, if the end_date value of the previous row (with the same num_ID) is at least 7 days away from the current start_date.

| num_ID | start_date  | end_date   | other_column      | total_times |
| ------ | ----------- | ---------- | ----------------- | ----------- |
| 1      | 2022-02-14  | 2022-02-15 | 09:23:00          | 0           |
| 1      | 2022-02-20  | 2022-02-25 | 12:10:01          | 0           |
| 2      | 2022-03-11  | 2022-03-21 | 08:21:00          | 0           |
| 2      | 2022-03-22  | 2022-03-27 | 02:36:00          | 0           |
| 2      | 2022-04-10  | 2022-04-15 | 11:43:03          | 1           |
| 3      | 2022-02-04  | 2022-02-06 | 16:51:00          | 0           |
| 3      | 2022-02-14  | 2022-02-23 | 19:35:10          | 1           |
| 4      | 2022-02-28  | 2022-10-12 | 00:01:00          | 0           |

CodePudding user response：

You can use a comparison shifted per group with groupby.shift and compare it to your threshold with ge (≥) and convert the boolean Series to integer:

df['total_times'] = (df['start_date'].sub(df.groupby('num_ID')['end_date'].shift())
                     .ge('7d').astype(int)
                    )

output:

   num_ID start_date   end_date other_column  total_times
0       1 2022-02-14 2022-02-15     09:23:00            0
1       1 2022-02-20 2022-02-25     12:10:01            0
2       2 2022-03-11 2022-03-21     08:21:00            0
3       2 2022-03-22 2022-03-27     02:36:00            0
4       2 2022-04-10 2022-04-15     11:43:03            1
5       3 2022-02-04 2022-02-06     16:51:00            0
6       3 2022-02-14 2022-02-23     19:35:10            1
7       4 2022-02-28 2022-10-12     00:01:00            0