I need to filter DF in pySpark to check if a cell’s value equals to the results of this cell’s XOR a number for example: df.num ^ xor_num == df.num Here is my code:

new_df = old_df.filter(old_df.num ^ xor_num == old.df.num)

But I get the error: unsupported operand type for ^:’column’ and ‘int’

(The reason is obvious, those are different types) So my question is - how can I use this filter method to make a bit operator on the data to compare it to itself?

There is a similar question in stack overflow:

Xor logical condition in pyspark

BUT it is not relevant and Does not solve my problem, because it is how to make a xor on the results and not on the cell input. Thanks for answering.

CodePudding user response：

As your xor_num is a integer, you can transfer the integer to column by lit:

new_df = old_df.filter(func.col('num').bitwiseXOR(func.lit(xor_num))==func.col('num'))