I am trying to host a web app in a container with read only file system. Whenever I try to configure the root file system as read only through the SecurityContext of the container I get the following error:
Ports: 80/TCP, 443/TCP
Host Ports: 0/TCP, 0/TCP
State: Terminated
Reason: Error
Exit Code: 137
Started: Thu, 23 Sep 2021 18:13:08 0300
Finished: Thu, 23 Sep 2021 18:13:08 0300
Ready: False
I've tried to achieve the same using an AppArmor profile as follows:
profile parser-profile flags=(attach_disconnected) {
#include <abstractions/base>
...
deny /** wl,
...
Unfortunately the result is the same.
What I assume is happening is that the container is not capable of saving the files for the web app and fails.
In my scenario, I will be running untrusted code and I must make sure that users are not allowed to access the file system.
Any ideas of what I am doing wrong and how can I achieve a read only file system?
I am using AKS and below is my deployment configuration:
apiVersion: v1
kind: Service
metadata:
name: parser-service
spec:
selector:
app: parser
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
- port: 443
targetPort: 443
protocol: TCP
name: https
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: parser-deployment
spec:
replicas: 5
selector:
matchLabels:
app: parser
template:
metadata:
labels:
app: parser
annotations:
container.apparmor.security.beta.kubernetes.io/parser: localhost/parser-profile
spec:
containers:
- name: parser
image: parser.azurecr.io/parser:latest
ports:
- containerPort: 80
- containerPort: 443
resources:
limits:
cpu: "1.20"
securityContext:
readOnlyRootFilesystem: true
Edit: I also tried creating a cluster level PSP which also did not work.
CodePudding user response:
I managed to replicate your issue and achieve read only filesystem with exception for one directory.
First, worth to note that you are using both solutions in your deployment - the AppArmor profile and SecurityContext. As AppArmor seems to be much more complex and needs configuration to be done per node I decided to use only SecurityContext as it is working fine.
I got this error that you mention in the comment:
Failed to create CoreCLR, HRESULT: 0x80004005
This error doesn't say to much, but after some testing I found that it only occurs when you are running the pod which filesytem is read only - the application tries to save files but cannot do so.
The app creates some files in the /tmp
directory so the solution is to mount /tmp
using Kubernetes Volumes so it will be read write. In my example I used emptyDir but you can use any other volume you want as long as it supports writing to it. The deployment configuration (you can see that I added volumeMounts
and volumes
and the bottom):
apiVersion: v1
kind: Service
metadata:
name: parser-service
spec:
selector:
app: parser
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
- port: 443
targetPort: 443
protocol: TCP
name: https
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: parser-deployment
spec:
replicas: 5
selector:
matchLabels:
app: parser
template:
metadata:
labels:
app: parser
spec:
containers:
- name: parser
image: parser.azurecr.io/parser:latest
ports:
- containerPort: 80
- containerPort: 443
resources:
limits:
cpu: "1.20"
securityContext:
readOnlyRootFilesystem: true
volumeMounts:
- mountPath: /tmp
name: temp
volumes:
- name: temp
emptyDir: {}
After executing into pod I can see that pod file system is mounted as read only:
# ls
app bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
# touch file
touch: cannot touch 'file': Read-only file system
# mount
overlay on / type overlay (ro,...)
By running kubectl describe pod {pod-name}
I can see that /tmp
directory is mounted as read write and it is using temp
volume:
Mounts:
/tmp from temp (rw)
Keep in mind if you are using other directories (for example to save files) you need to also mount them the same way as the /tmp
.