less_equal not working like I expect (pyarrow.compute.less

I'm trying to debug this issue I'm having with pyarrow. See this code snippet:

pa_execution_date = Z['execution_date'][i]
py_execution_date = pa_execution_date.as_py()
pa_report_date = Z['report_date'][i]
py_report_date = pa_report_date.as_py()
print(pa_execution_date)
print(pa_report_date)
print(py_execution_date)
print(py_report_date)
assert (pc.less_equal(pa_execution_date, pa_report_date))
assert (py_execution_date <= py_report_date)

What I'm seeing is that the second assertion is failing but not the first (in some cases). This is really odd because the two comparison operations should be equivalent...

Here's the output from the printout when this happens:

1591303729000000000
1591303728000000000
1591303729000000000
1591303728000000000

Any ideas about what I'm doing wrong?

I was expecting the first assertion to fail before the second assertion has a chance to execute and fail. I was not expecting the second assertion to fail without the first assertion failing first.

CodePudding user response：

I found the problem. The expression pc.less_equal(pa_execution_date, pa_report_date) actually returns a BooleanScalar, a pyarrow object, rather than a python bool. Adding .as_py() does the trick.

CodePudding user response：

pyarrow.compute.less_equal returns a pyarrow.BooleanScaler object. That's not a false-like value as var as Python is concerned. You can do this to convert it to a Python value:

assert pc.less_equal(pa_execution_date, pa_report_date).as_py()