Airflow Elasticsearch Provider doesn't work with HTTPS schema-CodePudding

When creating a new connection in Airflow 2.3.3 with the elasticsearch 7.17.2 provider the 'test' is never successful when using https, but if enable HTTP access to my elastic cluster it works fine. I added the root CA cert to the the docker containers via ca-certificates and when i docker exec to airflow-airflow-worker-1 and curl https://{url} I get the expected result ( no SSL errors ), but the airflow UI connection test continues to fail.

Any suggestions? Thanks in advance

CodePudding user response：

You can install the pre-release version (4.1.0rc1) of the elasticsearch provider which allows an es_conn_args option that can be used to configure the location of the CA certificate for secure connections to Elastic search.

import certifi
from airflow.providers.elasticsearch.hooks import ElasticsearchPythonHook

eshook = ElasticsearchPythonHook(
    hosts=["https://localhost:9200"], 
    es_conn_args={'ca_cert': certifi.where()}
)

Otherwise, you can subclass the ElasticsearchHook in your current version to override the get_conn method and pass ca_cert option in the es.elastic.api.connect function.