I have a RPI4 with 32-bit base image, for now I need to use 32bit, so armv7l architecture.
I want to run jupyterlab on it and do some data science. I am also new to docker, so please guide me gently.
For that I created a Dockerfile jupyterlab and some other packages:
FROM python:3.9.9-slim-buster
WORKDIR /
COPY wheels ./wheels
## INSTALL WITH APK
RUN apt-get update && apt-get install --no-cache -y \
g \
gcc \
python3-dev \
# jpeg-dev \
# zlib-dev \
# python3-pip \
make \
wget \
libatlas-base-dev \
libffi-dev
## INSTALL WITH PIP
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir setuptools && \
pip install --no-cache-dir cffi && \
#pip install --no-cache-dir matplotlib && \
pip install --no-cache-dir /wheels/numpy-1.21.4-cp39-cp39-linux_armv7l.whl && \
pip install --no-cache-dir /wheels/scipy-1.7.2-cp39-cp39-linux_armv7l.whl && \
pip install --no-cache-dir /wheels/Pillow-8.1.2-cp39-none-any.whl && \
pip install --no-cache-dir /wheels/pandas-1.3.4-cp39-cp39-linux_armv7l.whl && \
pip install --no-cache-dir /wheels/matplotlib-3.5.0-cp39-cp39-linux_armv7l.whl && \
#pip install --no-cache-dir /wheels/matplotlib-3.4.3-cp39-cp39-linux_armv7l.whl && \
pip install --no-cache-dir /wheels/jupyterlab-4.0.0a15-py3-none-any.whl
# Does not work in Dockerfile, but afterwards
RUN pip install --upgrade --force-reinstall matplotlib && \
pip install --upgrade --force-reinstall pandas
...
After I created a container from the image, I cannot use matplotlib and pandas. It misses libjpeg and some other dependencies, I couldn't install. After reinstalling matplotlib and pandas I finally can import them. So why I cannot write this into my Dockerfile, but can run it inside the container in console? This in this line I get an error:
# Does not work in Dockerfile, but afterwards
RUN pip install --upgrade --force-reinstall matplotlib && \
pip install --upgrade --force-reinstall pandas
Here is are the build logs of docker:
docker buildx build --platform linux/arm/v7 -t jupyter:latestBUSTER --load .
[ ] Building 2026.5s (9/14)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.75kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3.9.9-slim-buster 2.0s
=> [ 1/11] FROM docker.io/library/python:3.9.9-slim-buster@sha256:1894f5a820074eef46c09cb88df 0.0s
=> => resolve docker.io/library/python:3.9.9-slim-buster@sha256:1894f5a820074eef46c09cb88df65 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 4.83kB 0.0s
=> CACHED [ 2/11] COPY wheels ./wheels 0.0s
=> CACHED [ 3/11] RUN apt-get update && apt-get install -y g make wget liba 0.0s
=> [ 4/11] RUN pip install --no-cache-dir --upgrade pip && pip install --no-cache-dir 1217.8s
=> ERROR [ 5/11] RUN pip install --upgrade --force-reinstall matplotlib pandas 806.5s
------
> [ 5/11] RUN pip install --upgrade --force-reinstall matplotlib pandas:
#9 5.621 Collecting matplotlib
#9 5.837 Downloading matplotlib-3.5.1.tar.gz (35.3 MB)
#9 19.99 Preparing metadata (setup.py): started
#9 35.15 Preparing metadata (setup.py): finished with status 'error'
#9 35.15 ERROR: Command errored out with exit status 1:
#9 35.15 command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-o8gmzlfp/matplotlib_57b5e7b28dea459a90ea3c2ac3ee7833/setup.py'"'"'; __file__='"'"'/tmp/pip-install-o8gmzlfp/matplotlib_57b5e7b28dea459a90ea3c2ac3ee7833/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-bfl4k4wv
#9 35.15 cwd: /tmp/pip-install-o8gmzlfp/matplotlib_57b5e7b28dea459a90ea3c2ac3ee7833/
#9 35.15 Complete output (90 lines):
#9 35.15
#9 35.15 Edit mplsetup.cfg to change the build options; suppress output with --quiet.
#9 35.15
#9 35.15 BUILDING MATPLOTLIB
#9 35.15 python: yes [3.9.9 (main, Dec 3 2021, 12:52:50) [GCC 8.3.0]]
#9 35.15 platform: yes [linux]
#9 35.15 tests: no [skipping due to configuration]
#9 35.15 macosx: no [Mac OS-X only]
#9 35.15
#9 35.15 running egg_info
#9 35.15 creating /tmp/pip-pip-egg-info-bfl4k4wv/matplotlib.egg-info
#9 35.15 writing /tmp/pip-pip-egg-info-bfl4k4wv/matplotlib.egg-info/PKG-INFO
#9 35.15 writing dependency_links to /tmp/pip-pip-egg-info-bfl4k4wv/matplotlib.egg-info/dependency_links.txt
#9 35.15 writing namespace_packages to /tmp/pip-pip-egg-info-bfl4k4wv/matplotlib.egg-info/namespace_packages.txt
#9 35.15 writing requirements to /tmp/pip-pip-egg-info-bfl4k4wv/matplotlib.egg-info/requires.txt
#9 35.15 writing top-level names to /tmp/pip-pip-egg-info-bfl4k4wv/matplotlib.egg-info/top_level.txt
#9 35.15 writing manifest file '/tmp/pip-pip-egg-info-bfl4k4wv/matplotlib.egg-info/SOURCES.txt'
#9 35.15 Traceback (most recent call last):
#9 35.15 File "/usr/local/lib/python3.9/site-packages/numpy/core/__init__.py", line 22, in <module>
#9 35.15 from . import multiarray
#9 35.15 File "/usr/local/lib/python3.9/site-packages/numpy/core/multiarray.py", line 12, in <module>
#9 35.15 from . import overrides
#9 35.15 File "/usr/local/lib/python3.9/site-packages/numpy/core/overrides.py", line 7, in <module>
#9 35.15 from numpy.core._multiarray_umath import (
#9 35.15 ImportError: /lib/arm-linux-gnueabihf/libm.so.6: version `GLIBC_2.29' not found (required by /usr/local/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-arm-linux-gnueabihf.so)
#9 35.15
#9 35.15 During handling of the above exception, another exception occurred:
#9 35.15
#9 35.15 Traceback (most recent call last):
#9 35.15 File "<string>", line 1, in <module>
#9 35.15 File "/tmp/pip-install-o8gmzlfp/matplotlib_57b5e7b28dea459a90ea3c2ac3ee7833/setup.py", line 273, in <module>
#9 35.15 setup( # Finally, pass this all along to setuptools to do the heavy lifting.
#9 35.15 File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
#9 35.15 return distutils.core.setup(**attrs)
#9 35.15 File "/usr/local/lib/python3.9/distutils/core.py", line 148, in setup
#9 35.15 dist.run_commands()
#9 35.15 File "/usr/local/lib/python3.9/distutils/dist.py", line 966, in run_commands
#9 35.15 self.run_command(cmd)
#9 35.15 File "/usr/local/lib/python3.9/distutils/dist.py", line 985, in run_command
#9 35.15 cmd_obj.run()
#9 35.15 File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 299, in run
#9 35.15 self.find_sources()
#9 35.15 File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
#9 35.15 mm.run()
#9 35.15 File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 541, in run
#9 35.15 self.add_defaults()
#9 35.15 File "/usr/local/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 578, in add_defaults
#9 35.15 sdist.add_defaults(self)
#9 35.15 File "/usr/local/lib/python3.9/distutils/command/sdist.py", line 228, in add_defaults
#9 35.15 self._add_defaults_ext()
#9 35.15 File "/usr/local/lib/python3.9/distutils/command/sdist.py", line 311, in _add_defaults_ext
#9 35.15 build_ext = self.get_finalized_command('build_ext')
#9 35.15 File "/usr/local/lib/python3.9/distutils/cmd.py", line 299, in get_finalized_command
#9 35.15 cmd_obj.ensure_finalized()
#9 35.15 File "/usr/local/lib/python3.9/distutils/cmd.py", line 107, in ensure_finalized
#9 35.15 self.finalize_options()
#9 35.15 File "/tmp/pip-install-o8gmzlfp/matplotlib_57b5e7b28dea459a90ea3c2ac3ee7833/setup.py", line 78, in finalize_options
#9 35.15 self.distribution.ext_modules[:] = [
#9 35.15 File "/tmp/pip-install-o8gmzlfp/matplotlib_57b5e7b28dea459a90ea3c2ac3ee7833/setup.py", line 81, in <listcomp>
#9 35.15 for ext in package.get_extensions()
#9 35.15 File "/tmp/pip-install-o8gmzlfp/matplotlib_57b5e7b28dea459a90ea3c2ac3ee7833/setupext.py", line 381, in get_extensions
#9 35.15 add_numpy_flags(ext)
#9 35.15 File "/tmp/pip-install-o8gmzlfp/matplotlib_57b5e7b28dea459a90ea3c2ac3ee7833/setupext.py", line 493, in add_numpy_flags
#9 35.15 import numpy as np
#9 35.15 File "/usr/local/lib/python3.9/site-packages/numpy/__init__.py", line 150, in <module>
#9 35.15 from . import core
#9 35.15 File "/usr/local/lib/python3.9/site-packages/numpy/core/__init__.py", line 48, in <module>
#9 35.15 raise ImportError(msg)
#9 35.15 ImportError:
#9 35.15
#9 35.15 IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
#9 35.15
#9 35.15 Importing the numpy C-extensions failed. This error can happen for
#9 35.15 many reasons, often due to issues with your setup or how NumPy was
#9 35.15 installed.
#9 35.15
#9 35.15 We have compiled some common reasons and troubleshooting tips at:
#9 35.15
#9 35.15 https://numpy.org/devdocs/user/troubleshooting-importerror.html
#9 35.15
#9 35.15 Please note and check the following:
#9 35.15
#9 35.15 * The Python version is: Python3.9 from "/usr/local/bin/python"
#9 35.15 * The NumPy version is: "1.21.4"
#9 35.15
#9 35.15 and make sure that they are the versions you expect.
#9 35.15 Please carefully study the documentation linked above for further help.
#9 35.15
#9 35.15 Original error was: /lib/arm-linux-gnueabihf/libm.so.6: version `GLIBC_2.29' not found (required by /usr/local/lib/python3.9/site-packages/numpy/core/_multiarray_umath.cpython-39-arm-linux-gnueabihf.so)
#9 35.15
#9 35.15 ----------------------------------------
#9 35.15 WARNING: Discarding https://files.pythonhosted.org/packages/8a/46/425a44ab9a71afd2f2c8a78b039c1af8ec21e370047f0ad6e43ca819788e/matplotlib-3.5.1.tar.gz#sha256=b2e9810e09c3a47b73ce9cab5a72243a1258f61e7900969097a817232246ce1c (from https://pypi.org/simple/matplotlib/) (requires-python:>=3.7). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
#9 35.22 Downloading matplotlib-3.5.0.tar.gz (35.0 MB)
#9 46.61 Preparing metadata (setup.py): started
#9 61.26 Preparing metadata (setup.py): finished with status 'error'
#9 61.26 ERROR: Command errored out with exit status 1:
#9 61.26 command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-o8gmzlfp/matplotlib_cbcfb5d58f91495e99292752b0a4b614/setup.py'"'"'; __file__='"'"'/tmp/pip-install-o8gmzlfp/matplotlib_cbcfb5d58f91495e99292752b0a4b614/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-gcp5p8dk
#9 61.26 cwd: /tmp/pip-install-o8gmzlfp/matplotlib_cbcfb5d58f91495e99292752b0a4b614/
#9 61.26 Complete output (90 lines):
#9 61.26
#9 61.26 Edit mplsetup.cfg to change the build options; suppress output with --quiet.
#9 61.26
.................................
#9 785.9 Downloading matplotlib-0.86.1.tar.gz (2.6 MB)
#9 788.3 Preparing metadata (setup.py): started
#9 789.7 Preparing metadata (setup.py): finished with status 'error'
#9 789.7 ERROR: Command errored out with exit status 1:
#9 789.7 command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-o8gmzlfp/matplotlib_a84ab2b5496c45688637895fdbd6d8b4/setup.py'"'"'; __file__='"'"'/tmp/pip-install-o8gmzlfp/matplotlib_a84ab2b5496c45688637895fdbd6d8b4/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-red4rtzy
#9 789.7 cwd: /tmp/pip-install-o8gmzlfp/matplotlib_a84ab2b5496c45688637895fdbd6d8b4/
#9 789.7 Complete output (6 lines):
#9 789.7 Traceback (most recent call last):
#9 789.7 File "<string>", line 1, in <module>
#9 789.7 File "/tmp/pip-install-o8gmzlfp/matplotlib_a84ab2b5496c45688637895fdbd6d8b4/setup.py", line 105
#9 789.7 print 'installing data to', datapath
#9 789.7 ^
#9 789.7 SyntaxError: Missing parentheses in call to 'print'. Did you mean print('installing data to', datapath)?
#9 789.7 ----------------------------------------
#9 789.7 WARNING: Discarding https://files.pythonhosted.org/packages/ba/7f/2405de5ef84e7069095c17efd7c6a4a4120b1a9054731ac231163705c3a9/matplotlib-0.86.1.tar.gz#sha256=886d0cc85c74370c6f7502d4b6a054fe32106283c8e94896675eca3dacce350e (from https://pypi.org/simple/matplotlib/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
#9 789.8 Downloading matplotlib-0.86.tar.gz (2.6 MB)
#9 792.1 Preparing metadata (setup.py): started
#9 793.6 Preparing metadata (setup.py): finished with status 'error'
#9 793.6 ERROR: Command errored out with exit status 1:
#9 793.6 command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-o8gmzlfp/matplotlib_7c34facfebde40a58712b239be4a94cd/setup.py'"'"'; __file__='"'"'/tmp/pip-install-o8gmzlfp/matplotlib_7c34facfebde40a58712b239be4a94cd/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vhs2f2lw
#9 793.6 cwd: /tmp/pip-install-o8gmzlfp/matplotlib_7c34facfebde40a58712b239be4a94cd/
#9 793.6 Complete output (6 lines):
#9 793.6 Traceback (most recent call last):
#9 793.6 File "<string>", line 1, in <module>
#9 793.6 File "/tmp/pip-install-o8gmzlfp/matplotlib_7c34facfebde40a58712b239be4a94cd/setup.py", line 105
#9 793.6 print 'installing data to', datapath
#9 793.6 ^
#9 793.6 SyntaxError: Missing parentheses in call to 'print'. Did you mean print('installing data to', datapath)?
#9 793.6 ----------------------------------------
#9 793.6 WARNING: Discarding https://files.pythonhosted.org/packages/09/1a/1015c9ec4b8b13b6c6edfa3fa3d2f0e05860f80414c7d80ccfc7787f1220/matplotlib-0.86.tar.gz#sha256=67e9c4d076c5920125ee5e85c654b3e00c81bceed1b8b2ec431e8133045093d9 (from https://pypi.org/simple/matplotlib/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
#9 793.6 ERROR: Could not find a version that satisfies the requirement matplotlib (from versions: 0.86, 0.86.1, 0.86.2, 0.91.0, 0.91.1, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.3.0, 1.3.1, 1.4.0, 1.4.1rc1, 1.4.1, 1.4.2, 1.4.3, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0b1, 2.0.0b2, 2.0.0b3, 2.0.0b4, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.0.1, 2.0.2, 2.1.0rc1, 2.1.0, 2.1.1, 2.1.2, 2.2.0rc1, 2.2.0, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 3.0.0rc2, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.1.0rc1, 3.1.0rc2, 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.2.0rc1, 3.2.0rc3, 3.2.0, 3.2.1, 3.2.2, 3.3.0rc1, 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4, 3.4.0rc1, 3.4.0rc2, 3.4.0rc3, 3.4.0, 3.4.1, 3.4.2, 3.4.3, 3.5.0b1, 3.5.0rc1, 3.5.0, 3.5.1)
#9 793.6 ERROR: No matching distribution found for matplotlib
------
Dockerfile:56
--------------------
54 | pip install --no-cache-dir /wheels/jupyterlab-4.0.0a15-py3-none-any.whl
55 |
56 | >>> RUN pip install --upgrade --force-reinstall matplotlib pandas
57 |
58 | #RUN update-alternatives --config x-www-browser
--------------------
error: failed to solve: process "/bin/sh -c pip install --upgrade --force-reinstall matplotlib pandas" did not complete successfully: exit code: 1
I will also provide a "pip freeze" of the container with matplotlib and panday additionally installed afterwards, maybe this helps:
# pip freeze
anyio==3.4.0
argon2-cffi==21.1.0
attrs==21.2.0
Babel==2.9.1
backcall==0.2.0
bleach==4.1.0
certifi==2021.10.8
cffi==1.15.0
charset-normalizer==2.0.7
cPython==0.0.6
cycler==0.11.0
debugpy==1.5.1
decorator==5.1.0
defusedxml==0.7.1
entrypoints==0.3
fonttools==4.28.3
glibc==0.6.1
idna==3.3
ipykernel==6.5.1
ipython==7.29.0
ipython-genutils==0.2.0
jedi==0.18.1
Jinja2==3.0.3
json5==0.9.6
jsonschema==4.2.1
jupyter-client==7.1.0
jupyter-core==4.9.1
jupyter-server==1.12.0
jupyterlab @ file:///wheels/jupyterlab-4.0.0a15-py3-none-any.whl
jupyterlab-pygments==0.1.2
jupyterlab-server==2.8.2
kiwisolver==1.3.2
MarkupSafe==2.0.1
matplotlib==3.5.1
matplotlib-inline==0.1.3
mistune==0.8.4
nbclassic==0.3.4
nbclient==0.5.9
nbconvert==6.3.0
nbformat==5.1.3
nest-asyncio==1.5.1
notebook==6.4.6
numpy==1.21.4
packaging==21.3
pandas==1.3.5
pandocfilters==1.5.0
parso==0.8.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.4.0
prometheus-client==0.12.0
prompt-toolkit==3.0.22
ptyprocess==0.7.0
pycparser==2.21
Pygments==2.10.0
pylibjpeg-libjpeg==1.2.0
pymongo==4.0.1
pyparsing==3.0.6
pyrsistent==0.18.0
python-dateutil==2.8.2
pytz==2021.3
pyzmq==22.3.0
requests==2.26.0
scipy @ file:///wheels/scipy-1.7.2-cp39-cp39-linux_armv7l.whl
Send2Trash==1.8.0
setuptools-scm==6.3.2
six==1.16.0
sniffio==1.2.0
terminado==0.12.1
testpath==0.5.0
tomli==1.2.2
tornado==6.1
traitlets==5.1.1
urllib3==1.26.7
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.2.1
These are some cmds I issued to partially solve the problem:
matplotlib:
pip install --upgrade --force-reinstall matplotlib:
Successfully installed cycler-0.11.0 fonttools-4.28.3 kiwisolver-1.3.2 matplotlib-3.5.1 numpy-1.21.4 packaging-21.3 pillow-8.4.0 pyparsing-3.0.6 python-dateutil-2.8.2 six-1.16.0
pillow:
pip install GLIBC
pandas:
AttributeError: partially initialized module 'pandas' has no attribute 'core' (most likely due to a circular import)
Thank you for the time and effort.
CodePudding user response：
I fixed it by writing matplotlib and pandas on top of the RUN command. Here is the fixed dockerfile:
FROM python:3.9.9-slim-buster
WORKDIR /
COPY wheels ./wheels
## INSTALL WITH APK
RUN apt-get update && apt-get install -y \
g \
gcc \
python3-dev \
libjpeg-dev \
zlib1g-dev \
make \
wget \
libatlas-base-dev \
libffi-dev
## INSTALL WITH PIP
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir pillow && \
pip install --no-cache-dir matplotlib && \
pip install --no-cache-dir pandas && \
pip install --no-cache-dir setuptools && \
pip install --no-cache-dir cffi && \
pip install --no-cache-dir GLIBC && \
pip install --no-cache-dir /wheels/numpy-1.21.4-cp39-cp39-linux_armv7l.whl && \
pip install --no-cache-dir /wheels/scipy-1.7.2-cp39-cp39-linux_armv7l.whl && \
pip install --no-cache-dir /wheels/jupyterlab-4.0.0a15-py3-none-any.whl
But now I will downgrade to python 3.7.12 or 3.6 to support a package manager like conda and some other packages like tensorflow and scikit-learn, which got no builds for armv7 and python3.9.