Ansible Elasticsearch Cluster Deployment

Elasticsearch is the backbone of the ELK/Elastic Stack, handling search, logging, and analytics at scale. Deploying Elasticsearch clusters manually is error-prone — node discovery, JVM tuning, shard allocation, and security all need precise configuration. Ansible makes this manageable.

Why Automate Elasticsearch with Ansible

Cluster topology is defined in inventory — add/remove nodes by editing a file
Rolling upgrades without cluster downtime
Consistent JVM and OS tuning across all nodes
Security configuration (TLS, users, roles) version-controlled
Index lifecycle policies deployed as code

Prerequisites

Ansible 2.14+ on controller
Target nodes: 4+ GB RAM minimum (8+ GB recommended)
Dedicated disks for data nodes
Network connectivity between all cluster nodes (ports 9200, 9300)

Cluster Architecture

┌─────────────────────────────────────────────┐
│           Elasticsearch Cluster             │
├─────────────┬──────────────┬───────────────┤
│  Master x3  │  Data x3+    │  Coord x2    │
│  (voting)   │  (storage)   │  (routing)   │
└─────────────┴──────────────┴───────────────┘

Inventory Structure

[elasticsearch:children]
es_masters
es_data
es_coordinators

[es_masters]
es-master-01 ansible_host=10.0.1.10
es-master-02 ansible_host=10.0.1.11
es-master-03 ansible_host=10.0.1.12

[es_data]
es-data-01 ansible_host=10.0.2.10 es_data_path=/data/elasticsearch
es-data-02 ansible_host=10.0.2.11 es_data_path=/data/elasticsearch
es-data-03 ansible_host=10.0.2.12 es_data_path=/data/elasticsearch

[es_coordinators]
es-coord-01 ansible_host=10.0.3.10
es-coord-02 ansible_host=10.0.3.11

[elasticsearch:vars]
es_cluster_name=production-logs
es_version=8.14
es_heap_size=4g
es_network_host=_site_

Main Deployment Playbook

---
- name: Deploy Elasticsearch Cluster
  hosts: elasticsearch
  become: true
  vars:
    es_major_version: 8
    es_package: "elasticsearch"
    es_config_dir: /etc/elasticsearch
    es_data_dir: "{{ es_data_path | default('/var/lib/elasticsearch') }}"
    es_log_dir: /var/log/elasticsearch
    es_transport_port: 9300
    es_http_port: 9200
    es_bootstrap_password: "{{ vault_es_bootstrap_password }}"

  tasks:
    - name: Configure OS limits for Elasticsearch
      ansible.builtin.template:
        src: elasticsearch-limits.conf.j2
        dest: /etc/security/limits.d/elasticsearch.conf
        mode: "0644"

    - name: Disable swap permanently
      ansible.builtin.lineinfile:
        path: /etc/fstab
        regexp: '.*swap.*'
        state: absent
      notify: Disable swap now

    - name: Set vm.max_map_count
      ansible.posix.sysctl:
        name: vm.max_map_count
        value: "262144"
        sysctl_set: true
        reload: true

    - name: Set vm.swappiness
      ansible.posix.sysctl:
        name: vm.swappiness
        value: "1"
        sysctl_set: true
        reload: true

    - name: Install Elasticsearch GPG key
      ansible.builtin.rpm_key:
        key: https://artifacts.elastic.co/GPG-KEY-elasticsearch
        state: present
      when: ansible_os_family == "RedHat"

    - name: Add Elasticsearch repository
      ansible.builtin.yum_repository:
        name: elasticsearch
        description: "Elasticsearch {{ es_major_version }}.x"
        baseurl: "https://artifacts.elastic.co/packages/{{ es_major_version }}.x/yum"
        gpgcheck: true
        gpgkey: https://artifacts.elastic.co/GPG-KEY-elasticsearch
      when: ansible_os_family == "RedHat"

    - name: Add Elasticsearch APT repository
      ansible.builtin.apt_repository:
        repo: "deb https://artifacts.elastic.co/packages/{{ es_major_version }}.x/apt stable main"
        state: present
      when: ansible_os_family == "Debian"

    - name: Install Elasticsearch
      ansible.builtin.package:
        name: "{{ es_package }}"
        state: present

    - name: Create data directory
      ansible.builtin.file:
        path: "{{ es_data_dir }}"
        state: directory
        owner: elasticsearch
        group: elasticsearch
        mode: "0755"

    - name: Deploy Elasticsearch configuration
      ansible.builtin.template:
        src: elasticsearch.yml.j2
        dest: "{{ es_config_dir }}/elasticsearch.yml"
        owner: root
        group: elasticsearch
        mode: "0660"
      notify: Restart Elasticsearch

    - name: Configure JVM heap size
      ansible.builtin.template:
        src: jvm.options.d/heap.options.j2
        dest: "{{ es_config_dir }}/jvm.options.d/heap.options"
        owner: root
        group: elasticsearch
        mode: "0660"
      notify: Restart Elasticsearch

    - name: Enable and start Elasticsearch
      ansible.builtin.systemd:
        name: elasticsearch
        enabled: true
        state: started

    - name: Wait for Elasticsearch to be ready
      ansible.builtin.uri:
        url: "https://localhost:{{ es_http_port }}/_cluster/health"
        user: elastic
        password: "{{ es_bootstrap_password }}"
        validate_certs: false
        status_code: 200
      register: cluster_health
      until: cluster_health.status == 200
      retries: 30
      delay: 10

  handlers:
    - name: Disable swap now
      ansible.builtin.command: swapoff -a

    - name: Restart Elasticsearch
      ansible.builtin.systemd:
        name: elasticsearch
        state: restarted

Elasticsearch Configuration Template

templates/elasticsearch.yml.j2:

cluster.name: {{ es_cluster_name }}
node.name: {{ inventory_hostname }}

# Node roles
node.roles:
{% if inventory_hostname in groups['es_masters'] %}
  - master
{% endif %}
{% if inventory_hostname in groups['es_data'] %}
  - data
  - data_content
  - data_hot
{% endif %}
{% if inventory_hostname in groups['es_coordinators'] %}
  - ingest
{% endif %}

path.data: {{ es_data_dir }}
path.logs: {{ es_log_dir }}

network.host: {{ es_network_host }}
http.port: {{ es_http_port }}
transport.port: {{ es_transport_port }}

# Discovery
discovery.seed_hosts:
{% for host in groups['es_masters'] %}
  - {{ hostvars[host]['ansible_host'] }}:{{ es_transport_port }}
{% endfor %}

cluster.initial_master_nodes:
{% for host in groups['es_masters'] %}
  - {{ host }}
{% endfor %}

# Security
xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12

xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Performance
indices.memory.index_buffer_size: 20%
thread_pool.write.queue_size: 1000

JVM Heap Configuration

templates/jvm.options.d/heap.options.j2:

-Xms{{ es_heap_size }}
-Xmx{{ es_heap_size }}

Index Lifecycle Management

- name: Configure ILM policies
  hosts: es_coordinators[0]
  tasks:
    - name: Create hot-warm-cold ILM policy
      ansible.builtin.uri:
        url: "https://localhost:9200/_ilm/policy/logs-lifecycle"
        method: PUT
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        body_format: json
        body:
          policy:
            phases:
              hot:
                min_age: "0ms"
                actions:
                  rollover:
                    max_primary_shard_size: "50gb"
                    max_age: "1d"
                  set_priority:
                    priority: 100
              warm:
                min_age: "3d"
                actions:
                  shrink:
                    number_of_shards: 1
                  forcemerge:
                    max_num_segments: 1
                  set_priority:
                    priority: 50
              cold:
                min_age: "30d"
                actions:
                  freeze: {}
                  set_priority:
                    priority: 0
              delete:
                min_age: "90d"
                actions:
                  delete: {}
        status_code: [200, 201]

    - name: Create index template with ILM
      ansible.builtin.uri:
        url: "https://localhost:9200/_index_template/logs-template"
        method: PUT
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        body_format: json
        body:
          index_patterns: ["logs-*"]
          template:
            settings:
              number_of_shards: 3
              number_of_replicas: 1
              index.lifecycle.name: logs-lifecycle
              index.lifecycle.rollover_alias: logs
          priority: 100
        status_code: [200, 201]

Rolling Cluster Upgrade

- name: Rolling Elasticsearch upgrade
  hosts: elasticsearch
  serial: 1
  become: true
  vars:
    es_new_version: "8.15.0"
  tasks:
    - name: Disable shard allocation
      ansible.builtin.uri:
        url: "https://localhost:9200/_cluster/settings"
        method: PUT
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        body_format: json
        body:
          persistent:
            cluster.routing.allocation.enable: "primaries"
      run_once: true
      delegate_to: "{{ groups['es_coordinators'][0] }}"

    - name: Flush synced
      ansible.builtin.uri:
        url: "https://localhost:9200/_flush/synced"
        method: POST
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        status_code: [200, 409]

    - name: Stop Elasticsearch
      ansible.builtin.systemd:
        name: elasticsearch
        state: stopped

    - name: Upgrade Elasticsearch package
      ansible.builtin.package:
        name: "elasticsearch-{{ es_new_version }}"
        state: present

    - name: Start Elasticsearch
      ansible.builtin.systemd:
        name: elasticsearch
        state: started

    - name: Wait for node to rejoin cluster
      ansible.builtin.uri:
        url: "https://localhost:9200/_cluster/health"
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        status_code: 200
      register: health
      until: health.json.status in ['green', 'yellow']
      retries: 60
      delay: 10

    - name: Re-enable shard allocation
      ansible.builtin.uri:
        url: "https://localhost:9200/_cluster/settings"
        method: PUT
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        body_format: json
        body:
          persistent:
            cluster.routing.allocation.enable: null

    - name: Wait for green cluster status
      ansible.builtin.uri:
        url: "https://localhost:9200/_cluster/health?wait_for_status=green&timeout=5m"
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        status_code: 200
      register: green_check
      until: green_check.json.status == 'green'
      retries: 30
      delay: 30

Snapshot and Backup

- name: Configure Elasticsearch backups
  hosts: es_coordinators[0]
  tasks:
    - name: Register S3 snapshot repository
      ansible.builtin.uri:
        url: "https://localhost:9200/_snapshot/s3_backup"
        method: PUT
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        body_format: json
        body:
          type: s3
          settings:
            bucket: "{{ es_backup_bucket }}"
            region: "{{ aws_region }}"
            base_path: "elasticsearch/snapshots"
            compress: true

    - name: Create SLM policy (daily snapshots)
      ansible.builtin.uri:
        url: "https://localhost:9200/_slm/policy/daily-snapshots"
        method: PUT
        user: elastic
        password: "{{ vault_es_bootstrap_password }}"
        validate_certs: false
        body_format: json
        body:
          schedule: "0 30 2 * * ?"
          name: "<daily-snap-{now/d}>"
          repository: s3_backup
          config:
            indices: ["*"]
            ignore_unavailable: true
            include_global_state: true
          retention:
            expire_after: "30d"
            min_count: 7
            max_count: 30

Troubleshooting

Problem	Cause	Solution
Cluster RED status	Unassigned primary shards	Check disk space, node connectivity
OutOfMemoryError	Heap too small or field data	Increase heap or add circuit breakers
Split brain	Network partition between masters	Ensure odd number of master-eligible nodes
Slow queries	Missing index mappings	Add explicit mappings, avoid wildcard queries
Disk watermark reached	Data node full	Add nodes or configure ILM to delete old data

Conclusion

Automating Elasticsearch with Ansible transforms a complex distributed system into a manageable, repeatable deployment. From initial cluster bootstrap to rolling upgrades and backup policies, every aspect of your search infrastructure becomes code.