How To Deploy Elasticsearch on Kubernetes: A Complete Guide

Elasticsearch is an open source search engine, used mostly for log analysis, monitoring, and full-text search. However, as businesses scale, managing Elasticsearch by itself can be tricky, this is where Kubernetes comes in. It helps manage complex servers that require high availability, scalability, and fault tolerance. 

Running Elasticsearch on Kubernetes allows you to employ container orchestration, automated scaling, and self-healing clusters. There are multiple ways to deploy Elasticsearch on Kubernetes and you can choose the one that best suits your requirements. 

In this guide, we will walk through the process of deploying Elasticsearch on Kubernetes, from prerequisites to setup! 

Prerequisites for Deploying Elasticsearch on Kubernetes

Before you start deploying Elasticsearch on Kubernetes, ensure that you have the following all set: 

• A running Kubernetes cluster 
• The Kubernetes command-line tool (kubectl) should be installed and configured to manage the cluster.
• Helm to simplify deploying Elasticsearch =/using prebuilt charts.
• Ensure that your system has a Persistent Volume (PV) or Persistent Volume Claim (PVC) configured for Elasticsearch data.
• Cluster  334rxsdzr mk,.Resources – At least 2–3 nodes with sufficient CPU and RAM for Elasticsearch pods.
• A dedicated namespace for Elasticsearch to keep resources organized.
• Proper role-based access control permissions to manage deployments and services.

Elasticsearch on Kubernetes Deployment Options

There are quite a few methods to deploy Elasticsearch on Kubernetes depending on your expertise level: 

• Manual Deployment with YAML Manifests

Create your own Kubernetes manifests to apply, such as Deployments, StatefulSets, Services, and PVCs. Manual deployment also allows full control over the process but it requires more effort. 

• Helm Charts

Using pre-set Helm charts, you can quickly deploy and manage Elasticsearch clusters on Kubernetes. It supports easy upgrades and scalability with little to no configuration. 

• Elastic Cloud on Kubernetes (ECK)

Using the official Kubernetes operator, you can easily manage Elasticsearch with automated scaling, upgrades, security, and monitoring. 

• Custom Kubernetes Operators

Use community operators or build your own for specific needs.

How to Deploy Elasticsearch on Kubernetes

Here is a step by step process to deploy Elasticsearch on Kubernetes. 

Step 1: Set Up a Namespace for Elasticsearch

Namespaces allow you to organize the resources efficiently and avoid conflicts. Create a specific namespace for Elasticsearch by running this command on kubectl.

kubectl create namespace elasticsearch

Verify by running:

kubectl get namespaces

Step 2: Configure Persistent Storage

Elasticsearch requires persistent storage for logs, you can define one using Persistent Volume Claim (PVC): 

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

  name: es-pvc

  namespace: elasticsearch

spec:

  accessModes:

    – ReadWriteOnce

  resources:

    requests:

      storage: 10Gi

Apply it:

kubectl apply -f es-pvc.yaml

Step 3: Deploy Elasticsearch Cluster on Kubernetes

You can deploy Elasticsearch on Kubernetes using a StatefulSet or Helm. Here is an example YAML: 

apiVersion: apps/v1

kind: StatefulSet

metadata:

  name: elasticsearch

  namespace: elasticsearch

spec:

  serviceName: “elasticsearch”

  replicas: 3

  selector:

    matchLabels:

      app: elasticsearch

  template:

    metadata:

      labels:

        app: elasticsearch

    spec:

      containers:

      – name: elasticsearch

        image: docker.elastic.co/elasticsearch/elasticsearch:8.15.0

        resources:

          requests:

            memory: 1Gi

            cpu: 500m

        ports:

        – containerPort: 9200

        volumeMounts:

        – name: es-storage

          mountPath: /usr/share/elasticsearch/data

  volumeClaimTemplates:

  – metadata:

      name: es-storage

    spec:

      accessModes: [“ReadWriteOnce”]

      resources:

        requests:

          storage: 10Gi

Apply:

kubectl apply -f elasticsearch-statefulset.yaml

Step 4: Expose Elasticsearch Service

Lastly, expose the Elasticsearch service so that it can be accessed from inside and outside the cluster. 

apiVersion: v1

kind: Service

metadata:

  name: elasticsearch

  namespace: elasticsearch

spec:

  ports:

  – port: 9200

    targetPort: 9200

  selector:

    app: elasticsearch

  type: ClusterIP

For external access, change type: ClusterIP to NodePort or LoadBalancer (if supported by your cloud provider).

Step 5: Access Elasticsearch from Inside and Outside the Cluster

Here is how you can access it from inside the cluster: 

Run a temporary pod and test connectivity:
kubectl run tmp-shell –rm -i –tty –image=alpine — sh

wget -qO- http://elasticsearch:9200

Here is how you can access it from outside the Cluster:

• If exposed as NodePort:
  curl http://<NodeIP>:<NodePort>
• If using LoadBalancer (cloud):
  curl http://<LoadBalancerIP>:9200

Related Article: What Is a Kubernetes DaemonSet? How to Create and Use DaemonSets Effectively

Configuring Elasticsearch on Kubernetes

Once deployed, you need to configure Elasticsearch for optimal performance, security, and cluster health. 

• Heap Size: Adjust JVM heap size to ~50% of available memory (avoid exceeding 32GB).
  – name: ES_JAVA_OPTS

  value: “-Xms2g -Xmx2g”

• Cluster Name & Node Roles: Define a unique cluster name and roles for all users. 
• Security: Enable the TLS and user authentication using the Elastic’s built-in security features. 
• Resource Requests/Limits: Allocate resource limits and requests in pod specifications. 
• Storage Class: Use the cloud provider’s high performing storage.

Elasticsearch Cloud on Kubernetes (ECK)

ECK also known as Elastic Cloud on Kubernetes is an operator that streamlines Elasticsearch deployment and management. It helps automate scaling, configuration, and upgrades. Since it also comes with TLS and RBAC, you do not need much manual intervention. 

YAML Example (minimal cluster):
apiVersion: elasticsearch.k8s.elastic.co/v1

kind: Elasticsearch

metadata:

  name: quickstart

spec:

  version: 8.15.0

  nodeSets:

  – name: default

    count: 3

    config:

•       node.store.allow_mmap: false

Common Issues and Troubleshooting

Issue	Cause	Solution
Pods stuck in Pending	PVC not bound or insufficient resources	Check kubectl describe pvc and ensure a valid StorageClass exists.
CrashLoopBackOff	JVM heap too large, misconfigured environment variables, or insufficient resources	Tune ES_JAVA_OPTS and adjust resource limits.
Cluster Red/Yellow Status	Not enough master/data nodes, or storage unavailable	Add more nodes, check PVCs, verify quorum.
Slow Performance	Limited CPU/memory, or default storage is too slow	Use SSD storage and allocate more resources.
Unable to access externally	Service type not exposed correctly	Change service type to NodePort or LoadBalancer.
Authentication errors	Security not configured properly	Enable built-in security, generate credentials with ECK or manual setup.

Conclusion – Elasticsearch on Kubernetes 

Deploying Elasticsearch on Kubernetes provides you with the right amount of stability, resilience, and flexibility to manage large-scale search and analytics workloads. For companies and organisations who need better automation for their clusters, this is the right way to do it! 

FAQs