I'm trying to find all containers that does not have any resources spec in the cluster and make an alert for that.
Now I'm thinking about use all container
and pod
label in kube_pod_container_info
query and do an absent
query for kube_pod_container_resource_requests
,but dont know how to achieve that.
CodePudding user response:
I found another way to meet my demand: container_memory_working_set_bytes{id=~".*besteffort.*",container!="POD",pod!=""}
all containers under this condition can be treated as "containers that does not have any resources spec"
CodePudding user response:
There is also the unless operator:
vector1
unlessvector2
results in a vector consisting of the elements ofvector1
for which there are no elements invector2
with exactly matching label sets. All matching elements in both vectors are dropped.
In the context of this question, you can take for example container_cpu_system_seconds_total
, which seems to be for any container, and container_spec_cpu_quota
which appears for the containers with a CPU limit set. So to show which containers have no CPU limit:
container_cpu_system_seconds_total{container!="POD",container=~". "}
unless on (pod, container) container_spec_cpu_quota
Finding which containers have no memory limit requires a different approach. If a container has no memory limit, its container_spec_memory_limit_bytes
metric is set to 0
. That is, the metric is always present and unless
won't help in this case, but and
will:
# the top line is the same as before
container_cpu_system_seconds_total{container!="POD",container=~". "}
and on (pod, container) container_spec_memory_limit_bytes == 0