I just installed prometheus operator as indicated here: https://github.com/prometheus-operator/kube-prometheus:
kubectl apply --server-side -f manifests/setup
kubectl wait \
--for condition=Established \
--all CustomResourceDefinition \
--namespace=monitoring
kubectl apply -f manifests/
After that I just tried to setup my own service monitor for grafana as follows:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: in1-grafana-service-monitor
namespace: monitoring
spec:
selector:
matchLabels:
app.kubernetes.io/name: grafana
endpoints:
- port: http
interval: 10s
This monitor works just fine and I can see it in the Prometheus /targets and /service-discovery.
The fact is that when I want to create this same service monitor but outside the "monitoring" namespace it just not appears neither in /targets or in /service-discovery. My setup for this service monitor is as follows:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: out1-grafana-service-monitor
namespace: other-namespace
spec:
selector:
matchLabels:
app.kubernetes.io/name: grafana
namespaceSelector:
any: true
endpoints:
- port: http
interval: 10s
How can I make Prometheus operator to scrape service monitors (and services) outside the monitoring namespace?
I checked the output of kubectl get prom -Ao yaml
and it just displays an empty list:
[...]
serviceMonitorNamespaceSelector: {}
serviceMonitorSelector: {}
[...]
Any help will be appreciated.
Thank you.
I expect that the service monitor outside the monitoring namespace works as I need it for other service (Not for Grafana).
CodePudding user response:
Can you maybe try to deploy it using the helm chart in another namespace: https://github.com/prometheus-community/helm-charts/tree/acd6ee3b87886ccd1dcdab1bbec684190a5055fa/charts/kube-prometheus-stack
and set prometheus.prometheusSpec.serviceMonitorSelectorNilUsesHelmValues
to false
(link) in the values.
Then check if it works with a serviceMonitor outside of the operator namespace. If it works compare the YAML with your operator and find the difference