How to add custom index using ingest node pipeline?-CodePudding

Is it possible to create conditional indexing by using ingest node pipelines? I feel this could be done by the script processor but can someone tell if this is possible?

I am in a scenario where I should decide which is a better way to do custom indexing. I can mention conditions in the metricbeat.yml /filebeat.yml files to get this done. But is this the best way to do custom indexing? There is no logstash in my elastic stack

output.elasticsearch:
          indices:
            - index: "metricbeat-dev-%{[agent.version]}-%{ yyyy.MM.dd}"
              when.equals:
                kubernetes.namespace: "dev"

This is how I have implemented custom indexing in metric/filebeat right now. I have like 20 namespaces in my Kubernetes cluster. Please help in suggesting if this could be done by ingest node pipeline or not

CodePudding user response：

Yes, You can achived this by ingest pipeline Set Processor. Ingest Pipeline support accessing of metadata fields and you can access / update index name using _index field name.

Below is sample Ingest Pipeline which will update index name when namespace is dev:

[
  {
    "set": {
      "field": "_index",
      "value": "metricbeat-dev",
      "if": "ctx.kubernetes?.namespace== 'dev'"
    }
  }
]