Ansible Forks — Parallel Execution and Performance

Q: How Forks Work?

With forks=20, all hosts run Task 1 simultaneously, then all move to Task 2.

Introduction

Ansible's forks setting controls how many hosts are managed simultaneously. The default is 5, meaning Ansible connects to 5 hosts at once, runs the task, then moves to the next 5. For large inventories, tuning forks (along with serial, throttle, and async) dramatically reduces execution time.

Setting Forks

ansible.cfg

# ansible.cfg
[defaults]
forks = 50

Command Line

# Override for a single run
ansible-playbook site.yml -f 50
ansible all -m ping -f 100

Environment Variable

export ANSIBLE_FORKS=50
ansible-playbook site.yml

How Forks Work

With forks=5 and 20 hosts:

Batch 1: [host01] [host02] [host03] [host04] [host05]  → Task 1
Batch 2: [host06] [host07] [host08] [host09] [host10]  → Task 1
Batch 3: [host11] [host12] [host13] [host14] [host15]  → Task 1
Batch 4: [host16] [host17] [host18] [host19] [host20]  → Task 1

Then ALL 20 hosts move to Task 2 (in batches of 5 again)

With forks=20, all hosts run Task 1 simultaneously, then all move to Task 2.

Choosing the Right Value

Inventory Size	Recommended Forks	Notes
1-10 hosts	5 (default)	No tuning needed
10-50 hosts	20-50	Match inventory size
50-200 hosts	50-100	Watch controller memory
200-1000 hosts	100-200	Enable pipelining
1000+ hosts	200-500	Use pull mode or AWX

Performance Tuning Checklist

# ansible.cfg — full performance config
[defaults]
forks = 100
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp/ansible-facts
fact_caching_timeout = 86400

[ssh_connection]
pipelining = True
ssh_args = -o ControlMaster=auto -o ControlPersist=60s -o PreferSharedConnections=no

SSH Pipelining

Reduces SSH operations per task from 3 to 1:

[ssh_connection]
pipelining = True

Requires requiretty disabled in /etc/sudoers on targets.

SSH Multiplexing

Reuse SSH connections:

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=600s
control_path_dir = /tmp/.ansible/cp

Fact Caching

Avoid re-gathering facts every run:

[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp/ansible-facts
fact_caching_timeout = 86400

Forks vs Serial vs Throttle

# These three control different aspects of parallelism:

---
- name: Understanding parallelism
  hosts: all             # 100 hosts
  serial: 20             # Process 20 hosts per batch
  # forks = 50 in ansible.cfg
  # → 20 hosts run in parallel (serial < forks, so serial wins)

  tasks:
    - name: Normal task
      ansible.builtin.debug:
        msg: "Running on {{ inventory_hostname }}"
      # Uses up to 20 parallel connections (limited by serial)

    - name: Rate-limited API call
      ansible.builtin.uri:
        url: "https://api.example.com/register"
        method: POST
      throttle: 3
      # Only 3 hosts call the API at once (within the batch of 20)

Setting	Controls	Scope
`forks`	Max parallel SSH connections	Global
`serial`	Hosts per batch in a play	Play-level
`throttle`	Max parallel executions of one task	Task-level
`async`	Non-blocking task execution	Task-level

Async Tasks

Run long tasks without blocking:

    - name: Long-running update
      ansible.builtin.apt:
        upgrade: dist
      async: 3600        # Max runtime: 1 hour
      poll: 0            # Don't wait (fire and forget)
      register: update_job

    - name: Do other work while updating
      ansible.builtin.debug:
        msg: "Doing other things..."

    - name: Wait for update to finish
      ansible.builtin.async_status:
        jid: "{{ update_job.ansible_job_id }}"
      register: job_result
      until: job_result.finished
      retries: 60
      delay: 30

Monitoring Performance

# Enable task profiling
export ANSIBLE_CALLBACKS_ENABLED=profile_tasks

# Run with timing info
ansible-playbook site.yml

# Output shows per-task timing:
# TASK [Install packages] ****
# ok: [web01] => (elapsed 12.3s)
# ok: [web02] => (elapsed 11.8s)

Controller Resource Usage

Memory per fork: ~50-100MB
CPU: minimal (SSH is I/O bound)

Formula:
  forks × 100MB = required RAM

Examples:
  50 forks  →  ~5 GB RAM
  100 forks → ~10 GB RAM
  200 forks → ~20 GB RAM

Troubleshooting

Issue	Solution
"Too many open files"	Increase ulimit: `ulimit -n 65535`
SSH connection timeouts	Reduce forks or increase `timeout`
Controller OOM killed	Reduce forks, enable fact caching
Tasks still slow	Enable pipelining, SSH multiplexing
"Unreachable" errors at high forks	Network/firewall rate limiting — reduce forks

Best Practices

Start conservative — increase forks gradually and monitor
Enable pipelining — biggest single performance improvement
Use fact caching — avoid gathering facts every run
Match forks to inventory — no benefit exceeding host count
Combine with serial — use serial for rolling updates, forks for raw speed
Monitor controller — watch RAM and open file descriptors

Conclusion

Tuning forks is the simplest way to speed up Ansible at scale. Combined with SSH pipelining, connection multiplexing, and fact caching, you can manage thousands of hosts efficiently. Start with forks=50, enable pipelining, and increase from there based on your controller's resources.