Sending data from elasticsearch to kafka and finally to influxdb?-CodePudding

I would like to know how can I send data from elasticsearch to kafka and then to influxdb? I've already tried using confluent platform with sources connector from elasticsearch and sink connector from influxdb, but the problem is that I'm stuck on sending data from elasticsearch to kafka

moreover once my computer is off I no longer have the backup of the connectors and I have to start from scratch

that's why my questions:

How to send data from elasticsearch to kafka? using confluent platform?
Do I really have to use confluent platform if I want to use kafka connect?

CodePudding user response：

Kafka Connect is Apache 2.0 Licensed and is included with Apache Kafka download.

Confluent (among other companies) write plugins for it, such as Sinks to Elasticsearch or Influx.

It appears the Elasticsearch source on Confluent Hub is not built by Confluent, for example.

once my computer is off I no longer have the backup of the connectors and I have to start from scratch

Kafka Connect distributed mode stores its config data in Kafka topics... Kafka defaults to store topic data in /tmp... Which is deleted when you shutdown your computer

Similarly, if you are using Docker for any of these systems without mounted volumes, Docker also is not persistent by default