Comparing dataframe columns with acceptable deviation in Python-CodePudding

How can I have the comparison between two pandas dataframe rows considering an acceptable deviation instead of a 100% match?

For example, with acceptable deviation = 10, I want to generate the Result column below based on the other columns:

column1 | column2 | Result
100     | 110     | True
0       | 20      | False
0       | 9       | True

I wasn't able to find any built-in functions or Pandas functions.

CodePudding user response：

Use:

df['Result'] = abs(df['column1'] - df['column2']) <= 10
print(df)

# Output:
   column1  column2  Result
0      100      110    True
1        0       20   False
2        0        9    True

Alternative, chained methods:

df['Result'] = df['column1'].sub(df['column2']).abs().le(10)

CodePudding user response：

One simple way:

c1 = df['column1']
c2 = df['column2']
dev = 10
df['Result'] = (c2 - c1 <= dev) & (c1 - c2 >= -dev)

Output:

>>> df
   column1  column2  Result
0      100      110    True
1        0       20   False
2        0        9    True

Another solution:

dev = 10
diff = df.diff(axis=1)['column2']
df['Result'] = (diff <= dev) & (diff >= -dev)

Output:

>>> df
   column1  column2  Result
0      100      110    True
1        0       20   False
2        0        9    True

CodePudding user response：

This should work. Using assign for use in pipelines.

acc_dev = 10

data = pd.DataFrame(
    [
        [100, 110, True],
        [0, 20, False],
        [0, 9, True],
    ],
    columns = ["column1", "column2", "Result"]
)

data.assign(Result=lambda d: abs(d["column1"] - d["column2"]) <= 10)