Apply a custom rolling function with arguments on Pandas DataFrame-CodePudding

I have this df (here is the df.head()):

    date        colA
0   2018-01-05  0.6191
1   2018-01-20  0.5645
2   2018-01-25  0.5641
3   2018-01-27  0.5404
4   2018-01-30  0.4933

I would like to apply a function to every 3 rows recursively, meaning for rows: 1,2,3 then for rows: 2,3,4 then rows 3,4,5, etc.

This is what I wrote:

def my_rolling_func(df, val):
  
    p1 = (df['date']-df['date'].min()).dt.days.tolist()[0],df[val].tolist()[0]
    p2 = (df['date']-df['date'].min()).dt.days.tolist()[1],df[val].tolist()[1]
    p3 = (df['date']-df['date'].min()).dt.days.tolist()[2],df[val].tolist()[2]
  
    return sum([i*j for i,j in [p1,p2,p3]])

df.rolling(3,center=False,axis=1).apply(my_rolling_func, args=('colA'))

But I get this error:

ValueError: Length of passed values is 1, index implies 494.

494 is the number of rows in my df.

I'm not sure why it says I passed a length of 1, I thought the rolling generate slices of df according to the window size I defined (3), and then it applied the function for that subset of df.

CodePudding user response：

First, you specified the wrong axis. Axis 1 means that the window will slide along the columns. You want the window to slide along the indexes, so you need to specify axis=0. Secondly, you misunderstand a little about how rolling works. It will apply your function to each column independently, so you cannot operate on both the date and colA columns at the same time inside your function. I rewrote your code to make it work:

import pandas as pd
import numpy as np

df = pd.DataFrame({'date':pd.date_range('2018-01-05', '2018-01-30', freq='D'), 'A': np.random.random((26,))})
df = df.set_index('date')


def my_rolling_func(s):
    days = (s.index - s.index[0]).days
    return sum(s*days)


res = df.rolling(3, center=False, axis=0).apply(my_rolling_func)

print(res)

Out: 
                  A
date                
2018-01-05       NaN
2018-01-06       NaN
2018-01-07  1.123872
2018-01-08  1.121119
2018-01-09  1.782860
2018-01-10  0.900717
2018-01-11  0.999509
2018-01-12  1.755408
2018-01-13  2.344914
       .....