Ansible Dynamic Inventory — Auto-Discover Cloud

Introduction

Dynamic inventory automatically discovers your infrastructure from cloud APIs, container platforms, and CMDBs. Instead of maintaining static host lists, Ansible queries AWS, Azure, GCP, Docker, or any API to build the inventory at runtime.

AWS EC2

# inventory/aws_ec2.yml
plugin: amazon.aws.aws_ec2
regions:
  - us-east-1
  - eu-west-1

keyed_groups:
  # Group by instance tags
  - key: tags.Environment
    prefix: env
    separator: "_"
  # Group by instance type
  - key: instance_type
    prefix: type

groups:
  # Custom group based on tags
  webservers: "'web' in tags.Role"
  dbservers: "'db' in tags.Role"

filters:
  instance-state-name: running
  "tag:ManagedBy": ansible

compose:
  ansible_host: public_ip_address
  ansible_user: "'ec2-user'"

# Install collection
ansible-galaxy collection install amazon.aws

# Set credentials
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=secret...

# Test inventory
ansible-inventory -i inventory/aws_ec2.yml --graph
ansible-inventory -i inventory/aws_ec2.yml --list

Azure

# inventory/azure_rm.yml
plugin: azure.azcollection.azure_rm
auth_source: auto

include_vm_resource_groups:
  - production
  - staging

keyed_groups:
  - key: tags.Environment
    prefix: env
  - key: location
    prefix: region

groups:
  webservers: "'web' in tags.Role"

compose:
  ansible_host: public_ipv4_addresses[0] | default(private_ipv4_addresses[0])
  ansible_user: "'azureuser'"

GCP

# inventory/gcp.yml
plugin: google.cloud.gcp_compute
projects:
  - my-project-id
zones:
  - us-central1-a
  - us-east1-b

filters:
  - status = RUNNING

keyed_groups:
  - key: labels.environment
    prefix: env
  - key: machine_type
    prefix: type

groups:
  webservers: "'web' in labels.role"

compose:
  ansible_host: networkInterfaces[0].accessConfigs[0].natIP | default(networkInterfaces[0].networkIP)

Docker

# inventory/docker.yml
plugin: community.docker.docker_containers

docker_host: unix:///var/run/docker.sock

keyed_groups:
  - key: docker_labels.environment
    prefix: env
  - key: docker_state
    prefix: state

filters:
  - status == "running"

compose:
  ansible_connection: community.docker.docker
  ansible_docker_extra_args: ""

How It Works

ansible-playbook -i inventory/aws_ec2.yml site.yml
                     │
                     ▼
            ┌────────────────┐
            │  Inventory      │
            │  Plugin runs    │
            │                 │
            │  Queries AWS    │
            │  EC2 API        │
            └────────┬───────┘
                     │
                     ▼
            ┌────────────────┐
            │  Returns hosts  │
            │  + groups       │
            │  + variables    │
            └────────┬───────┘
                     │
                     ▼
            ┌────────────────┐
            │  Playbook runs  │
            │  against        │
            │  discovered     │
            │  hosts          │
            └────────────────┘

Key Concepts

keyed_groups

Auto-create groups from host attributes:

keyed_groups:
  # tag:Environment=production → group: env_production
  - key: tags.Environment
    prefix: env
    separator: "_"
    
  # instance_type=t3.micro → group: type_t3_micro  
  - key: instance_type
    prefix: type

compose

Set host variables from instance attributes:

compose:
  ansible_host: public_ip_address
  ansible_user: "'ubuntu'"
  ansible_port: 22
  app_tier: tags.get('Tier', 'unknown')

groups

Define groups using conditions:

groups:
  webservers: "'web' in tags.Role"
  production: "tags.Environment == 'production'"
  large_instances: "instance_type.startswith('m5')"

filters

Limit which instances are included:

# AWS
filters:
  instance-state-name: running
  "tag:ManagedBy": ansible

# GCP
filters:
  - status = RUNNING
  - labels.managed_by = ansible

Multiple Inventories

# Use directory — all files inside are loaded
ansible-playbook -i inventory/ site.yml

# inventory/ directory:
inventory/
├── static_hosts.yml       # Static hosts
├── aws_ec2.yml           # AWS dynamic
├── azure_rm.yml          # Azure dynamic
└── group_vars/
    └── all.yml

Combine Static + Dynamic

# inventory/static.yml — manually managed hosts
all:
  children:
    on_premise:
      hosts:
        bare-metal-01:
          ansible_host: 192.168.1.100
        bare-metal-02:
          ansible_host: 192.168.1.101

# inventory/aws_ec2.yml — auto-discovered
plugin: amazon.aws.aws_ec2
...

Both inventories merge when using -i inventory/ directory.

Cache for Performance

# Cache inventory to avoid API calls every run
# ansible.cfg
[inventory]
cache = true
cache_plugin = jsonfile
cache_connection = /tmp/ansible-inventory-cache
cache_timeout = 3600    # 1 hour

Custom Inventory Script

For APIs without a plugin, write a script:

#!/usr/bin/env python3
"""Custom dynamic inventory script."""
import json
import requests

def get_inventory():
    # Query your API
    response = requests.get("https://cmdb.example.com/api/servers")
    servers = response.json()
    
    inventory = {
        "_meta": {"hostvars": {}},
        "webservers": {"hosts": []},
        "dbservers": {"hosts": []},
    }
    
    for server in servers:
        name = server["hostname"]
        inventory["_meta"]["hostvars"][name] = {
            "ansible_host": server["ip"],
            "ansible_user": "deploy",
        }
        if server["role"] == "web":
            inventory["webservers"]["hosts"].append(name)
        elif server["role"] == "db":
            inventory["dbservers"]["hosts"].append(name)
    
    return inventory

if __name__ == "__main__":
    print(json.dumps(get_inventory(), indent=2))

chmod +x inventory.py
ansible-playbook -i inventory.py site.yml

Troubleshooting

"No hosts matched"

# Debug inventory
ansible-inventory -i inventory/aws_ec2.yml --list
ansible-inventory -i inventory/aws_ec2.yml --graph

# Check specific host
ansible-inventory -i inventory/aws_ec2.yml --host web-01

Slow Inventory

Enable caching in ansible.cfg:

[inventory]
cache = true
cache_plugin = jsonfile
cache_connection = /tmp/ansible-cache
cache_timeout = 300

AWS Credentials

# Use environment variables
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...

# Or AWS profile
export AWS_PROFILE=production

# Or IAM role (on EC2 instances)
# No config needed — auto-detected

Conclusion

Dynamic inventory eliminates manual host management. Use inventory plugins (aws_ec2, azure_rm, gcp_compute, docker_containers) to auto-discover infrastructure from cloud APIs. Define groups with keyed_groups (from tags/labels), set connection variables with compose, and filter instances with filters. Combine static and dynamic inventories in a directory. Enable caching for performance. Dynamic inventory is essential for cloud-native automation where instances are ephemeral.