FutureWarning pandas append concat-CodePudding

This is my Code:

class Signals:

def __init__(self,df, lags):
    self.df = df
    self.lags = lags

def gettrigger(self):
    dfx = pd.DataFrame()
    for i in range(self.lags   1):
        mask = (self.df['%K'].shift(i) < 20 ) & (self.df['%D'].shift(i) < 20)
        dfx = dfx.append(mask, ignore_index=True)
    return dfx.sum(axis=0)

def decide(self):
    self.df['trigger'] = np.where(self.gettrigger(), 1, 0)
    self.df['Buy'] = np.where((self.df.trigger) &
(self.df['%K'].between(20,80)) &  (self.df['%D'].between(20,80)) & (self.df.rsi > 50) & (self.df.macd > 0), 1, 0)

And after I execute this:

inst.decide()

Im getting this warning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead. dfx = dfx.append(mask, ignore_index=True)

Can anyone help me ? Im new in Python and I have a hard time to finde the solution

THANKS

CodePudding user response：

you are using the append() method to add two dataframes together. The warning is saying that this method is now deprecated. You can still use it and ignore the error, or use the new concat() method instead.

just replace the word append with concat and that's it.

CodePudding user response：

pandas is simply directing you to use the pd.concat() function instead of df.append(). You can check out what warnings are from here. Here's the api documentation of pd.concat

CodePudding user response：

If I replace append with concat i got this:

AttributeError: 'DataFrame' object has no attribute 'concat'

Can I not just shut this message off ?

CodePudding user response：

This warning is telling you that pandas has done away with the .append method (see doc).

Deprecation, in its programming sense, is the process of taking older code and marking it as no longer being useful within the codebase, usually because it has been superseded by newer code. The deprecated code is not immediately removed from the codebase because doing so may cause regression errors. (source)

The warning is helpful because it gives you a suggestion of an alternative function concat (see doc) to use.

You may revise your code as follows:


def __init__(self,df, lags):
    self.df = df
    self.lags = lags

def gettrigger(self):
    dfx = pd.DataFrame()
    for i in range(self.lags   1):
        mask = (self.df['%K'].shift(i) < 20 ) & (self.df['%D'].shift(i) < 20)
        dfx = pd.concat(objs=[dfx, mask], ignore_index=True)
    return dfx.sum(axis=0)

def decide(self):
    self.df['trigger'] = np.where(self.gettrigger(), 1, 0)
    self.df['Buy'] = np.where((self.df.trigger) &
(self.df['%K'].between(20,80)) &  (self.df['%D'].between(20,80)) & (self.df.rsi > 50) & (self.df.macd > 0), 1, 0)

CodePudding user response：

so you have a loop:

dfx = pd.DataFrame()
for i in range(self.lags   1):
    mask = (self.df["%K"].shift(i) < 20 ) & (self.df["%D"].shift(i) < 20)
    dfx = dfx.append(mask, ignore_index=True)

where you're accumulatively building a dataframe through append calls. This is inefficient! It has to build dataframes over and over, which is a costly process. And .append is getting deprecated anyway.

You should rather build a list of dataframes in the loop, and at the end concatanete them:

frames = []                    # <- switch to building a list
for i in range(self.lags   1):
   mask = (self.df["%K"].shift(i) < 20 ) & (self.df["%D"].shift(i) < 20)
   frames.append(mask)        # <- save the new frames in that list

dfx = pd.concat(frames)       # <- at the end, concatanate them

(Note that the concatanation logic is after the loop; directly replacing .append calls inside the loop with pd.concat is inefficient too because re-building issue again.)

You can use a list comprehension or a generator expression to the same effect, depending on how complex your loop body is.

Above for example, can be written as:

dfx = pd.concat((self.df["%K"].shift(i) < 20 ) & (self.df["%D"].shift(i) < 20)
                for i in range(self.lags   1))