How can i deploy elasticsearch on kubernetes cluster?-CodePudding

How can I deploy elasticsearch on kubernetes cluster ?? I need to use the elasticsearch out-side to my cluster ?? for that , which service that I should use ?? can any one post the deployment and service yaml file ?

CodePudding user response：

You can use the helm chart to deploy the elasticsearch if you want to run it in production.

Helm chart : https://github.com/elastic/helm-charts

If you are just deploying for development and testing you can below YAML file :

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app : elasticsearch
    component: elasticsearch
    release: elasticsearch
  name: elasticsearch
spec:
  podManagementPolicy: Parallel
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app : elasticsearch
      component: elasticsearch
      release: elasticsearch
  serviceName: elasticsearch
  template:
    metadata:
      creationTimestamp: null
      labels:
        app : elasticsearch
        component: elasticsearch
        release: elasticsearch
    spec:
      containers:
      - env:
        - name: cluster.name
          value: <SET THIS>
        - name: discovery.type
          value: single-node
        - name: ES_JAVA_OPTS
          value: -Xms512m -Xmx512m
        - name: bootstrap.memory_lock
          value: "false"
        image: elasticsearch:6.5.0
        imagePullPolicy: IfNotPresent
        name: elasticsearch
        ports:
        - containerPort: 9200
          name: http
          protocol: TCP
        - containerPort: 9300
          name: transport
          protocol: TCP
        resources:
          limits:
            cpu: 250m
            memory: 1Gi
          requests:
            cpu: 150m
            memory: 512Mi
        securityContext:
          privileged: true
          runAsUser: 1000
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /usr/share/elasticsearch/data
          name: elasticsearch-data
      dnsPolicy: ClusterFirst
      initContainers:
      - command:
        - sh
        - -c
        - chown -R 1000:1000 /usr/share/elasticsearch/data
        - sysctl -w vm.max_map_count=262144
        - chmod 777 /usr/share/elasticsearch/data
        - chomod 777 /usr/share/elasticsearch/data/node
        - chmod g rwx /usr/share/elasticsearch/data
        - chgrp 1000 /usr/share/elasticsearch/data
        image: busybox:1.29.2
        imagePullPolicy: IfNotPresent
        name: set-dir-owner
        resources: {}
        securityContext:
          privileged: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /usr/share/elasticsearch/data
          name: elasticsearch-data
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 10
  updateStrategy:
    type: OnDelete
  volumeClaimTemplates:
  - metadata:
      creationTimestamp: null
      name: elasticsearch-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi

Ref Gist : https://gist.github.com/harsh4870/ccd6ef71eaac2f09d7e136307e3ecda6

which service that I should use

You can expose the Elasticsearch service with type LoadBalancer and expose it to internet and use it.

service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: eks-srv
spec:
  selector:
    app: elasticsearch
    component: elasticsearch
  ports:
    - name: db
      protocol: TCP
      port: 9200
      targetPort: 9200
    - name: monitoring
      protocol: TCP
      port: 9300
      targetPort: 9300
  type: LoadBalancer

Note :

you need to use the NodePort or LoadBalancer as service type with ClusterIp you wont be able to expose service unless you use some proxy setup or ingress.

CodePudding user response：

You can try using YAML file as belwo:

kind: StatefulSet
apiVersion: apps/v1
metadata:
  name: elasticsearch
  labels:
    com/es: es
spec:
  replicas: 1
  selector:
    matchLabels:
          com/es: es
  template:
     metadata:
      labels:
        com/es: es
    spec:
      containers:
        - resources: {}
          terminationMessagePath: /dev/termination-log
          name: elasticsearch
          env:
            - name: discovery.type
              value: single-node
          ports:
            - name: rest
              containerPort: 9200
              protocol: TCP
            - name: inter-node
              containerPort: 9300
              protocol: TCP
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: <volume-name>
              mountPath: <mount-path>
          terminationMessagePolicy: File
          image: 'docker.elastic.co/elasticsearch/elasticsearch:7.10.2'
      restartPolicy: Always
      terminationGracePeriodSeconds: 10
      dnsPolicy: ClusterFirst
      securityContext: {}
      imagePullSecrets:
        - name: <image-pullsecret>
      schedulerName: default-scheduler
  volumeClaimTemplates:
    - kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: es-data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 15Gi
        volumeMode: Filesystem
  serviceName: es-service
  podManagementPolicy: OrderedReady

You can use this yaml which creates statefulset, statefullset will internally create the elaticsearch pod. If you want volume mount you can add your volume mount data which is mentioned in this yaml. if you don't delete the volume section from the spec and volumeClaimTemplates.

To use the elasticsearch out-side to cluster, try this:

kind: Service
apiVersion: v1
metadata:
  name: es-service
  labels:
    com/es: es
spec:
  ports:
    - name: rest
      protocol: TCP
      port: 9200
      targetPort: 9200
    - name: inter-node
      protocol: TCP
      port: 9300
      targetPort: 9300
  selector:
    com/es: es
  clusterIP: <ip of your cluster>
  clusterIPs:
    -  <ip of your cluster>
  type: ClusterIP

  ipFamilies:
    - IPv4

CodePudding user response：

As other answers have pointed out, you can use helm charts, however Elastic has also published its own operator which is a significantly more robust option than deploying a bare statefulSet

To install the operator:


kubectl create -f https://download.elastic.co/downloads/eck/2.0.0/crds.yaml
kubectl apply -f https://download.elastic.co/downloads/eck/2.0.0/operator.yaml

And to deploy a cluster


cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 8.0.0
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
EOF

If you want to have this production ready, you probably want to make some further adjustments that you can all find in the documentation