Ansible until Retry — Retry Tasks Until Success

Introduction

Some tasks fail temporarily — APIs return 503 during deploys, services take time to start, DNS propagates slowly. Ansible's until loop retries a task until a condition is met or retries are exhausted. This is essential for reliable automation in environments where transient failures are normal.

Basic Syntax

---
- name: Retry examples
  hosts: all
  tasks:
    - name: Wait for API to become available
      ansible.builtin.uri:
        url: "https://api.example.com/health"
        status_code: 200
      register: result
      until: result.status == 200
      retries: 30        # Maximum attempts
      delay: 10          # Seconds between attempts
      # Total max wait: 30 × 10 = 300 seconds (5 minutes)

Common Patterns

Wait for Service to Start

    - name: Start application
      ansible.builtin.systemd:
        name: myapp
        state: started

    - name: Wait for application to be ready
      ansible.builtin.uri:
        url: "http://localhost:8080/health"
        status_code: 200
      register: health
      until: health.status == 200
      retries: 30
      delay: 5

Wait for Port to Open

    - name: Wait for PostgreSQL to accept connections
      ansible.builtin.wait_for:
        host: "{{ db_host }}"
        port: 5432
        state: started
        timeout: 300
      # wait_for has built-in retry — but until is useful for more complex checks

    - name: Wait for database to be queryable
      community.postgresql.postgresql_query:
        db: myapp
        query: "SELECT 1"
      register: db_check
      until: db_check is succeeded
      retries: 30
      delay: 5

Retry Flaky API Calls

    - name: Create DNS record (API sometimes 503s)
      ansible.builtin.uri:
        url: "https://dns.example.com/api/v1/records"
        method: POST
        body_format: json
        body:
          name: "{{ hostname }}"
          type: A
          value: "{{ ip_address }}"
        status_code: [200, 201]
        headers:
          Authorization: "Bearer {{ api_token }}"
      register: dns_result
      until: dns_result.status in [200, 201]
      retries: 5
      delay: 15
      no_log: true

Wait for Cloud Instance

    - name: Launch EC2 instance
      amazon.aws.ec2_instance:
        name: webserver
        instance_type: t3.micro
        image_id: ami-12345678
        state: running
      register: ec2

    - name: Wait for SSH to be available
      ansible.builtin.wait_for:
        host: "{{ ec2.instances[0].public_ip_address }}"
        port: 22
        delay: 30
        timeout: 300

    - name: Wait for instance to pass status checks
      amazon.aws.ec2_instance_info:
        instance_ids:
          - "{{ ec2.instances[0].instance_id }}"
      register: instance_info
      until: >
        instance_info.instances[0].state.name == 'running' and
        instance_info.instances[0].state_transition_reason == ''
      retries: 30
      delay: 10

Wait for File to Appear

    - name: Wait for deployment marker file
      ansible.builtin.stat:
        path: /opt/app/DEPLOY_COMPLETE
      register: deploy_marker
      until: deploy_marker.stat.exists
      retries: 60
      delay: 5

Wait for Command Output

    - name: Wait for cluster to reach quorum
      ansible.builtin.command:
        cmd: rabbitmqctl cluster_status
      register: cluster
      until: "'running_nodes' in cluster.stdout and cluster.stdout.count('rabbit@') >= 3"
      retries: 30
      delay: 10
      changed_when: false

    - name: Wait for Kubernetes pod to be ready
      ansible.builtin.command:
        cmd: kubectl get pod myapp-0 -o jsonpath='{.status.phase}'
      register: pod_status
      until: pod_status.stdout == "Running"
      retries: 60
      delay: 5
      changed_when: false

Advanced Conditions

Multiple Conditions (AND)

    - name: Wait for healthy response with correct content
      ansible.builtin.uri:
        url: "http://localhost:8080/health"
        return_content: true
      register: health
      until:
        - health.status == 200
        - "'healthy' in health.content"
        - health.elapsed < 2
      retries: 30
      delay: 5

Complex Expressions

    - name: Wait for enough replicas
      ansible.builtin.command:
        cmd: kubectl get deployment myapp -o jsonpath='{.status.readyReplicas}'
      register: replicas
      until: (replicas.stdout | default('0') | int) >= 3
      retries: 60
      delay: 5
      changed_when: false

Retry with Rescue (Fallback)

    - name: Deployment with retry and fallback
      block:
        - name: Deploy new version
          ansible.builtin.command:
            cmd: /opt/deploy.sh v2.0
          register: deploy
          until: deploy.rc == 0
          retries: 3
          delay: 30
      rescue:
        - name: Rollback on failure
          ansible.builtin.command:
            cmd: /opt/deploy.sh rollback

Default Values

Parameter	Default	Description
`retries`	3	Number of retry attempts
`delay`	5	Seconds between retries
`until`	(required)	Condition to check

Calculating Timeouts

Total maximum wait time = retries × delay

Examples:
  retries: 30, delay: 10  →  5 minutes
  retries: 60, delay: 5   →  5 minutes
  retries: 12, delay: 30  →  6 minutes
  retries: 3,  delay: 5   → 15 seconds (default)

Troubleshooting

Issue	Solution
Task fails after all retries	Increase `retries` or `delay`; check if service actually starts
"FAILED - RETRYING" floods output	Normal; use `-v` for less noise or callback plugin
Condition never matches	Debug with `ansible.builtin.debug` to see actual values
Retry on wrong condition	Ensure `register` variable matches `until` expression
Want to retry on exception	Use `until: result is succeeded` for any-failure retry

Best Practices

Set realistic timeouts — retries × delay should match expected startup time
Use changed_when: false on check commands — polling doesn't change state
Prefer wait_for for ports — built-in, no register/until needed
Log the final state — debug the register variable after the until loop
Combine with block/rescue — retry first, then fall back on total failure
Don't retry destructive tasks — only retry idempotent/read-only operations

Conclusion

until turns fragile automation into resilient automation. Services need time to start, APIs have transient failures, and cloud resources take time to provision. Instead of adding arbitrary sleep tasks, until actively checks for the desired state and retries intelligently. Use it wherever you'd otherwise write "wait and hope."