Could not build wheels for pyarrow - Python 3.9-CodePudding

Recently, we've been seeing failures building pyarrow dependencies in our Python 3.9 project. This started about 08:57:01 PM, Oct 5 2021

25hCollecting pyarrow 20:58:39 Downloading pyarrow-5.0.0.tar.gz (739 kB) 20:58:39 |
████████████████████████████████| 739 kB 90.7 MB/s 21:00:1425h Installing build dependencies ... 25l- \ | / - \ | / - \ | / - error 21:00:14 ERROR: Command errored out 
with exit status 1: 21:00:14 command: /usr/local/bin/python /tmp/pip-standalone-pip-
55h74pun/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-
build-env-i45zzwqe/overlay --no-warn-script-location --no-binary :none: --only-binary 
:none: -i https://pypi.org/simple -- 'cython >= 0.29' 'numpy==1.16.6; 
python_version<'"'"'3.9'"'"'' 'numpy==1.19.4; python_version>='"'"'3.9'"'"'' setuptools 
setuptools_scm wheel 21:00:14 cwd: None

...

ERROR: Failed building wheel for pyarrow which use PEP 517 and cannot be installed directly
21:12:46

Before this time, we were building successfully, and we have made no changes to our code.

I do notice that our current jobs are failing on downloading pyarrow-5.0.0.tar.gz (739 kB) while the older, successful jobs were downloading pyarrow-5.0.0-cp39-cp39-manylinux2014_x86_64.whl (23.7 MB)

I am curious Why there was there a change from using a .whl file to a tar.gz file requirements.txt:

boto3
halo
pandas
numpy
pyarrow
s3fs
click

How we install:

pip install -r requirements.txt

I have attached a full output error log on the Pyarrow JIRA

Thank you for reading!

CodePudding user response：

You are not using python 3.9. You are using python 3.10. Note your install line...

command: /usr/local/bin/python /tmp/pip-standalone-pip-55h74pun/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-i45zzwqe/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'cython >= 0.29' 'numpy==1.16.6; python_version<'"'"'3.9'"'"'' 'numpy==1.19.4; python_version>='"'"'3.9'"'"'' setuptools setuptools_scm wheel

In particular I see python_version<3.9 and python_version>=3.9. Then a little further down:

  Ignoring numpy: markers 'python_version < "3.9"' don't match your environment

Finally, the wheels that are being downloaded are cp310 wheels.

You can track Python 3.10 support for pyarrow here: ARROW-14217