remove sub-field on demand from _source field during ingestion in elasticsearch 7.15-CodePudding

I have very big data field encoded in base64 that is causing me trouble during ingestion and I want to remove it from the _source but I cant , how I can achieve this but preserve other fields for this index in the _source field?

I have tried with the remove processor in the ingestion pipeline but without luck:

 {
   remove:{ field:"_source.data"}
 }

example document:

POST _ingest/pipeline/attachment/_simulate
  {
  "docs": [
  {
  "_index": "myindex",
  "_id": "pdf_doc",
  "_source": {
      "data": "1e1xydGYxXGFuc2kNCkxvgrgecmVtIGlwc3VtfrreIGRvbG9yIHNpdCBhbWVgr0DQpccGFyIH0="
  }
 }
]
}

CodePudding user response：

The syntax is documented here -

https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html#include-exclude