I want to use Prometheus to monitor Spark (using spark driver API) but I also want to use Kibana for better investigation capabilities. So I want to export those metrics from Prometheus also to Elastic Search as records to show on Kibana.

Is it somehow possible?

CodePudding user response：

You can check this blog where they have shown various way to export prometheus metrics to Elasticsearch.

You can use metricbeat as well to get data from prometheus as it provide module for same.

Also, if you are using latest version of Elasticsearch then you can explore Elastic Agent and Fleet as well, which have integration for prometheus.