Filebeat 7.12.1 ECK operator 2.2

I'm trying to setup the filbeat for the Nginx-ingress access logs in my ECK stack (installed in GKE). I can access the logs directly in the pod but nothing is coming to my Kibana dashboard. I have set up two filebeat.autodiscover.providers

1. hints.enabled: true, which looks for all the containers with co.elastic.logs/enabled: "true"
2. Checks the container containing name ingress. I can confirm that the name of the pod is nginx-ingress-ingress-nginx-controller-xxxx-xxxxx

Below is my Filebeat auto discover content:

apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: filebeat
  namespace: search
spec:
  type: filebeat
  version: 7.12.1
  elasticsearchRef:
    name: elastic-search
  kibanaRef:
    name: kibana-web
  config:
    filebeat.autodiscover.providers:
    - node: ${NODE_NAME}
      type: kubernetes
      hints.enabled: true
      #add_resource_metadata.namespace.enabled: true
      hints.default_config.enabled: "false"
    - node: ${NODE_NAME}
      type: kubernetes
      #add_resource_metadata.namespace.enabled: true
      hints.default_config.enabled: "false"
      templates:
      - condition:
          contains: 
            kubernetes.container.name: ingress
        config:
        - paths: ["/var/log/containers/*${data.kubernetes.container.id}.log"]
          type: container
          exclude_lines: ["^\\s [\\-`('.|_]"]
    processors:
    - add_cloud_metadata: {}
    - add_host_metadata: {}
  daemonSet:
    podTemplate:
      spec:
        serviceAccountName: filebeat
        automountServiceAccountToken: true
        terminationGracePeriodSeconds: 30
        dnsPolicy: ClusterFirstWithHostNet
        #hostNetwork: true # Allows to provide richer host metadata
        containers:
        - name: filebeat
          securityContext:
            runAsUser: 0
            # If using Red Hat OpenShift uncomment this:
            #privileged: true
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
          env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          resources:
            requests:
              memory: 200Mi
              cpu: 0.2
            limits:
              memory: 300Mi
              cpu: 0.4
              
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers

CodePudding user response：

Adding the answer here in case someone else run into this issue.

The issue is how I'm checking the contains condition. It should've been kubernetes.pod.name instead of kubernetes.container.name. So I replaced

- condition:
          contains: 
            kubernetes.container.name: ingress

to

- condition:
          contains: 
            kubernetes.pod.name: ingress

in the above file and things started to work!