Pandas group-by proportion of cumulative sum start from 0-CodePudding

I have the following pandas Data Frame (without 2 the last columns):

   name        day   show-in-appointment    previous-missed-appointments   proportion-previous-missed
0  Jack   2020/01/01   show                              0                      0
1  Jack   2020/01/02   no-show                           0                      0
2  Jill   2020/01/02   no-show                           0                      0
3  Jack   2020/01/03   show                              1                      0.5
4  Jill   2020/01/03   show                              1                      1
5  Jill   2020/01/04   no-show                           1                      0.5
6  Jack   2020/01/04   show                              1                      0.33
7  Jill   2020/01/05   show                              2                      0.66
8  jack   2020/01/06   no-show                           1                      0.25
9  jack   2020/01/07   show                              2                 0.4>>>2(noshow)/5(noshow show)

df = pd.DataFrame(
    data=np.asarray([
        ['Jack', 'Jack', 'Jill', 'Jack', 'Jill', 'Jill', 'Jack', 'Jill', 'jack', 'jack'],
        [
            '2020/01/01',
            '2020/01/02',
            '2020/01/02',
            '2020/01/03',
            '2020/01/03',
            '2020/01/04',
            '2020/01/04',
            '2020/01/05',
            '2020/01/06',
            '2020/01/07',
        ],
        ['show', 'no-show', 'no-show', 'show', 'show', 'no-show', 'show', 'show', 'no-show', 'show'],
    ]).T,
    columns=['name', 'day', 'show-in-appointment'],
)

previous-missed-appointments column is create like the code below:

df.name = df.name.str.capitalize()
df['order'] = df.index
df.day = pd.to_datetime(df.day)
df['noshow'] = df['show-in-appointment'].map({'show': 0, 'no-show': 1})
df = df.sort_values(by=['name', 'day'])
df['previous-missed-appointments'] = df.groupby('name').noshow.cumsum()
df.loc[df.noshow == 1, 'previous-missed-appointments'] -= 1
df = df.sort_values(by='order')
df = df.drop(columns=['noshow', 'order'])

********THE QUESTION IS

HOW CAN I CREATE THE LAST COLUMN ????*********

CodePudding user response：

You can use cumsum and shift in groupby.apply for the first column, then divide by groupby.cumcount for the second column:

m = df['show-in-appointment'].eq('no-show')

g = m.groupby(df['name'].str.casefold(), group_keys=False)
df['previous-missed-appointments'] =  (
  g.apply(lambda x: x.cumsum().shift(fill_value=0))
 )

df['proportion-previous-missed'] = (
    df['previous-missed-appointments'].div(g.cumcount()).fillna(0)
)

print(df)

Output:

   name         day show-in-appointment  previous-missed-appointments  proportion-previous-missed
0  Jack  2020/01/01                show                             0                    0.000000
1  Jack  2020/01/02             no-show                             0                    0.000000
2  Jill  2020/01/02             no-show                             0                    0.000000
3  Jack  2020/01/03                show                             1                    0.500000
4  Jill  2020/01/03                show                             1                    1.000000
5  Jill  2020/01/04             no-show                             1                    0.500000
6  Jack  2020/01/04                show                             1                    0.333333
7  Jill  2020/01/05                show                             2                    0.666667
8  jack  2020/01/06             no-show                             1                    0.250000
9  jack  2020/01/07                show                             2                    0.400000