I am using Alpine Linux as Docker image for my Python app in CI/CD (Jenkins). This is for AWS Lambda, basically the only noteworthy requirement is aws-psycopg2
for accessing Postgres. Locally everything works, it deploys nicely with Serverless and works on AWS, the only problem is on Jenkins. I am getting an error there:
16:24:55 ImportError: Error loading shared library libresolv.so.2: No such file or directory (needed by /usr/local/lib/python3.9/site-packages/psycopg2/../psycopg2_binary.libs/libgssapi_krb5-497db0c6.so.2.2)
I am NOT using Oracle in any way, like e.g. in this question.
My jenkinsfile.groovy
:
def testInstallStage = {
stage('Install') {
sh 'apk add python3-dev gcc libc-dev musl-dev openblas gfortran build-base postgresql-libs postgresql-dev libffi-dev cargo'
sh 'pip install --upgrade pip && pip install poetry'
sh 'poetry export --without-hashes --dev -f requirements.txt -o requirements.txt --with-credentials'
sh 'pip install -r requirements.txt'
sh 'pip install awscli'
sh 'aws configure set aws_access_key_id default_access_key'
sh 'aws configure set aws_secret_access_key default_secret_key'
sh 'aws configure set default.region eu-west-1'
sh 'cp config/jenkins.json config.json'
}
}
def testStage = {
stage('Tests') {
sh 'pip install nose && nosetests'
}
}
new pl.ruby.PipelineRunner(this, notificationChannel).run() {
pythonStack([image: '3.9.13-alpine3.16'], {
container('python') {
testInstallStage.call()
testStage.call()
}
})
}
The whole line with apk add
is from me trying to fix this. I tried installing many different dependencies. What should I install to get this working? I know that this is some library missing on Alpine, since on Ubuntu 20.04 and on AWS Lambda runtime it works ok.
CodePudding user response:
The library libresolv.so.2
is part of the system C library. Alpine-based images use a different version of the C library (musl libc) which is much smaller, but also occasionally runs into compatibility issues like this.
The easiest way around this is to switch to a Debian- or Ubuntu-based image based on the more standard GNU libc. Since you're using an unmodified image here you'd just need to remove the "Alpine" reference in the image tag
pythonStack([image: '3.9.13'], ...)
In a custom Dockerfile you might also need to change the package manager invocations if you need to install any OS-level packages, since Debian/Ubuntu and Alpine have different packaging formats, tools, and names. The first line of your testInstallStage
would also be affected by this.
The typical reason to use an Alpine-based image is to save space. The actual space savings here are fairly small (tens of megabytes) and once you start installing C header files the images quickly get much larger than this. You're probably not going to notice a practical difference in performance or image size making this change.