Is testing in Gitlab Ci supported with Opensearch docker images as a service?-CodePudding

I have a Java app that is running integration tests with Elasticsearch in Gitlab.

.gitlab-ci.yml:

...

integration:
  stage: integration
  tags:
    - onprem
  services:
    - name: "docker.elastic.co/elasticsearch/elasticsearch:7.10.1"
      alias: "elasticsearch"
      command: [ "bin/elasticsearch", "-Expack.security.enabled=false", "-Ediscovery.type=single-node" ]
  script:
    - curl "http://elasticsearch:9200/_cat/health"
    - mvn -Dgroups="IntegrationTest" -DargLine="-Durl=elasticsearch" test

...

Now I want to use Opensearch 1.1.0 because that is what we use on AWS. I tried working off the docker compose setup that Opensearch suggests for developers ( https://opensearch.org/docs/latest/opensearch/install/docker/#sample-docker-compose-file-for-development ), and came up with this:

...

integration:
  stage: integration
  tags:
    - onprem
  services:
    - name: "opensearchproject/opensearch:1.1.0"
      alias: "elasticsearch"
      command: [
          "./opensearch-docker-entrypoint.sh",
          "-Ecluster.name=opensearch-cluster",
          "-Enode.name=opensearch-node1",
          "-Ebootstrap.memory_lock=true",
          "-Ediscovery.type=single-node",
          "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m",
          "DISABLE_INSTALL_DEMO_CONFIG=true",
          "DISABLE_SECURITY_PLUGIN=true"
      ]
  script:
    - curl "http://elasticsearch:9200/_cat/health"
    - mvn -Dgroups="IntegrationTest" -DargLine="-Durl=elasticsearch" test

...

The curl response:

$ curl "http://elasticsearch:9200/_cat/health"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0curl: (7) Failed to connect to elasticsearch port 9200: No route to host

One big difference seems to be that Elasticsearch disables security with an environment variable, but Opensearch does that with an argument through setup. I tried running Opensearch directly through the "bin/" directory, but that seems to give all sorts of additional errors. The Opensearch image is available on dockerhub ( https://hub.docker.com/layers/opensearchproject/opensearch/1.1.0/images/sha256-94254d215845723e73829f34cf7053ae9810db360cf73c954737a717e9cf031d?context=explore ) , but I have no access to the Dockerfile of the Elasticsearch image to compare.

I have numerous other failed setups: Tried moving different combinations of the arguments over as stage variables in gitlab-ci.

Am I misunderstanding what to do here, or is what I'm trying even supported at all?

CodePudding user response：

After taking a look at the opensearch image https://hub.docker.com/r/opensearchproject/opensearch

I see statements like these

curl -XGET https://localhost:9200 -u admin:admin --insecure

Here it specifies localhost, it leads me to believe that if you don't specify network host that's what you will get

Taking a look at their docker-compose

https://github.com/flavienbwk/opensearch-docker-compose/blob/main/docker-compose.yml

You can specify the network.host

network.host: "0.0.0.0"

Elasticsearch needs this entry if you want to access it outside the localmachine i.e.

curl "http://elasticsearch:9200/_cat/health"

CodePudding user response：

The final layer in opensearchproject/opensearch:1.1.0 is

CMD ["./opensearch-docker-entrypoint.sh"]

which reconfigures opensearch based on environment variables, like DISABLE_SECURITY_PLUGIN, and populates opensearch startup options like -Eplugins.security.disabled=true

Even though such ENVs are accepted by Docker, bash does not allow export of environment variables like discovery.type=single-node, so the gitlab-ci job fails.

The CMD is refactored to ENTRYPOINT in later releases by this issue

One way to trick the CMD, is to set the variables before launching the script:

integration:
  stage: integration
  variables:
    OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"
    DISABLE_INSTALL_DEMO_CONFIG: "true"
    DISABLE_SECURITY_PLUGIN: "true"
  services:
    - name: opensearchproject/opensearch:1.1.0
      alias: opensearch
      command: ["bash", "-c", "env 'discovery.type=single-node' 'cluster.name=opensearch' ./opensearch-docker-entrypoint.sh"]
  script:
    - curl -sS http://opensearch:9200/_cat/health