I'm currently trying to bulk index a bunch of documents to a data stream in elastic using Python and I'm getting strange errors.
Here is my code:
for event in raw:
tmp = {
"@timestamp": event['timestamp'],
"event" : {"kwh": event['kwh']},
"location": f"{self.args.location}",
"sourcetype": "meterdata",
"host.name": f"{self.args.name}"
}
payload={
"_op_type": "create",
"_index": "energiemonitoring",
"_source": tmp
}
payloadlist.append(payload.copy())
payload=json.loads(json.dumps(payloadlist,default=self.make_json_serial))
r = helpers.bulk(
self.es_client,
payload,
raise_on_error=False
)
Which is following the documentation provided by Elastic here.
Sadly I get the following error:
elasticsearch.BadRequestError: BadRequestError(400, 'illegal_argument_exception', 'Malformed action/metadata line [1], expected field [create], [delete], [index] or [update] but found [_op_type]'
Replacing the _op_type with create is not helping either: Code:
payload={
"create": tmp,
"_index": "energiemonitoring",
}
Error:
elasticsearch.BadRequestError: BadRequestError(400, 'illegal_argument_exception', 'Action/metadata line [1] contains an unknown parameter [@timestamp]'
My Elastic version is 8.5.3
Has someone experienced a similar issue before?
Thank in advance!
CodePudding user response:
The serialization messed up the request. Thanks again for the help