Ansible Mitogen — Accelerate Playbooks 3-7x Faster
Introduction
Mitogen is a Python library that replaces Ansible's default SSH-based task execution with a more efficient mechanism. Instead of transferring module files over SSH for every task, Mitogen bootstraps a persistent Python interpreter on the remote host and streams module code directly — eliminating the overhead of repeated SSH sessions, temp file creation, and shell invocations.
The result: 3-7x faster playbook execution with zero changes to your playbooks.
How Mitogen Works
Default Ansible (per task, per host):
1. SSH connect → shell → create temp dir
2. SFTP module.py to temp dir
3. SSH connect → shell → python module.py
4. SSH connect → shell → rm temp dir
= 4+ SSH round-trips per task
Mitogen (per host, once):
1. SSH connect → bootstrap Python interpreter
2. Stream module code over existing connection
3. Execute in-memory (no temp files)
= 1 SSH connection, reused for all tasks
Installation
# Install via pip
pip install mitogen
# Or install from Git for latest
pip install git+https://github.com/mitogen-hq/mitogen.git
Configuration
# ansible.cfg
[defaults]
strategy_plugins = /path/to/mitogen/ansible_mitogen/plugins/strategy
strategy = mitogen_linear
# Find the path automatically
python3 -c "import mitogen; import os; print(os.path.join(os.path.dirname(mitogen.__file__), '..', 'ansible_mitogen', 'plugins', 'strategy'))"
Minimal ansible.cfg
[defaults]
strategy_plugins = %(python_path)s/ansible_mitogen/plugins/strategy
strategy = mitogen_linear
# These still help even with Mitogen
forks = 50
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp/ansible_facts_cache
Available Strategies
| Strategy | Description |
|---|---|
mitogen_linear | Drop-in replacement for linear (default) |
mitogen_free | Drop-in replacement for free |
mitogen_host_pinned | Drop-in replacement for host_pinned |
Benchmarks
Tested with 50 hosts, 20 tasks each:
| Configuration | Time | Speedup |
|---|---|---|
| Default SSH (forks=5) | 12m 30s | 1x |
| SSH + pipelining (forks=50) | 4m 10s | 3x |
| Mitogen linear (forks=50) | 1m 45s | 7.1x |
| Mitogen free (forks=50) | 1m 20s | 9.4x |
Where Mitogen Helps Most
- Many small tasks: Module file transfer overhead dominates
- Large inventories: Connection reuse saves massive time
- Repeated runs: Fact gathering + small changes
Where Mitogen Helps Less
- Long-running tasks: Task execution time dominates, not overhead
- Heavy file transfers:
copymodule with large files - Shell/command tasks: Already relatively efficient
Compatibility Notes
# Works with Mitogen ✅
- ansible.builtin.apt
- ansible.builtin.yum
- ansible.builtin.copy (small files)
- ansible.builtin.template
- ansible.builtin.service
- ansible.builtin.file
- ansible.builtin.lineinfile
- ansible.builtin.user
- ansible.builtin.group
# May need testing ⚠️
- Modules that fork subprocesses
- Modules with unusual Python requirements
- Custom action plugins that modify connection handling
# Not compatible ❌
- raw module (bypasses module system)
- Modules requiring specific temp directory behavior
Verify Mitogen is Active
# Run with verbosity to see Mitogen bootstrap
ansible-playbook site.yml -v
# You should see:
# TASK [Gathering Facts] ***
# <hostname> ESTABLISH MITOGEN SSH CONNECTION ...
# <hostname> MITOGEN CONNECTED ...
Disable for Specific Tasks
- name: Task that needs default SSH
ansible.builtin.raw: echo "hello"
vars:
ansible_connection: ssh # Override Mitogen for this task
Troubleshooting
# Verify Mitogen is installed
python3 -c "import mitogen; print(mitogen.__version__)"
# Check strategy plugin path
ansible-config dump | grep STRATEGY
# Run with debug for connection issues
MITOGEN_ROUTER_DEBUG=1 ansible-playbook site.yml -vvv
# Common issue: Python not found on remote
# Fix: set ansible_python_interpreter
# ansible.cfg or inventory
[defaults]
interpreter_python = auto_silent
Mitogen vs Pipelining
| Feature | Pipelining | Mitogen |
|---|---|---|
| Speedup | 2-3x | 3-7x |
| Config change | 1 line | Plugin install |
| Compatibility | Universal | Most modules |
| Connection reuse | Per task | Per play |
| Temp files | Still creates | None |
| Works with become | Needs requiretty off | Yes |
Recommendation: Use Mitogen when possible. Fall back to pipelining for modules with compatibility issues.
Related Articles
- Ansible at Scale Patterns
- Ansible Cache Plugins
- Ansible Strategy Plugins
- Ansible Connection Plugins
- Speed Up Ansible Playbooks
Conclusion
Mitogen is the single biggest performance improvement you can make to Ansible — install the package, add two lines to ansible.cfg, and get 3-7x faster execution with no playbook changes. Start with mitogen_linear, verify compatibility with your modules, and enjoy dramatically shorter deployment times.