Ansible Dynamic Inventory — AWS EC2, Azure, GCP

Introduction

Static inventory files don't work in cloud environments where instances are created and destroyed constantly. Dynamic inventory plugins query your cloud provider's API in real-time, automatically discovering hosts and their metadata. This guide covers AWS EC2, Azure, GCP, and custom scripts.

AWS EC2 Dynamic Inventory

Setup

# Install the AWS collection
ansible-galaxy collection install amazon.aws

# Requires boto3
pip install boto3 botocore

Configuration

# inventory/aws_ec2.yml (filename must end in aws_ec2.yml)
plugin: amazon.aws.aws_ec2
regions:
  - us-east-1
  - eu-west-1

# Authentication (or use env vars / instance profile)
aws_access_key_id: "{{ lookup('env', 'AWS_ACCESS_KEY_ID') }}"
aws_secret_access_key: "{{ lookup('env', 'AWS_SECRET_ACCESS_KEY') }}"

# Filter instances
filters:
  instance-state-name: running
  "tag:Environment": production

# Group by tags
keyed_groups:
  - key: tags.Environment
    prefix: env
    separator: "_"
  - key: tags.Role
    prefix: role
    separator: "_"
  - key: placement.region
    prefix: region
    separator: "_"
  - key: instance_type
    prefix: type
    separator: "_"

# Host variables from EC2 metadata
compose:
  ansible_host: public_ip_address | default(private_ip_address)
  ansible_user: "'ec2-user'"

# Hostnames
hostnames:
  - tag:Name
  - private-dns-name
  - instance-id

Test It

# List discovered hosts
ansible-inventory -i inventory/aws_ec2.yml --list

# Graph view
ansible-inventory -i inventory/aws_ec2.yml --graph
# @all:
#   |--@env_production:
#   |  |--web-server-01
#   |  |--web-server-02
#   |--@role_database:
#   |  |--db-primary

# Ping all discovered hosts
ansible -i inventory/aws_ec2.yml all -m ping

Azure Dynamic Inventory

# Install Azure collection
ansible-galaxy collection install azure.azcollection
pip install azure-identity azure-mgmt-compute azure-mgmt-network

# inventory/azure_rm.yml (filename must end in azure_rm.yml)
plugin: azure.azcollection.azure_rm
auth_source: auto

# Filter by resource group
include_vm_resource_groups:
  - production-rg
  - staging-rg

# Exclude powered-off VMs
conditional_groups:
  running: powerstate == "running"

# Group by tags and location
keyed_groups:
  - key: tags.Environment | default('untagged')
    prefix: env
  - key: location
    prefix: location
  - key: os_profile.system | default('linux')
    prefix: os

# Set connection variables
compose:
  ansible_host: public_ip_addresses[0] | default(private_ip_addresses[0])
  ansible_user: tags.ansible_user | default('azureuser')

GCP Dynamic Inventory

# Install GCP collection
ansible-galaxy collection install google.cloud
pip install google-auth google-api-python-client

# inventory/gcp.yml (filename must end in gcp.yml)
plugin: google.cloud.gcp_compute
projects:
  - my-project-id

auth_kind: application  # Uses GOOGLE_APPLICATION_CREDENTIALS

# Filter
filters:
  - status = RUNNING
  - labels.environment = production

# Group by labels and zone
keyed_groups:
  - key: labels.role
    prefix: role
  - key: zone
    prefix: zone

compose:
  ansible_host: networkInterfaces[0].accessConfigs[0].natIP | default(networkInterfaces[0].networkIP)

Enable Inventory Plugins

# ansible.cfg
[inventory]
enable_plugins = amazon.aws.aws_ec2, azure.azcollection.azure_rm, google.cloud.gcp_compute, host_list, script, auto, yaml, ini, toml

Combine Static and Dynamic

inventory/
├── static_hosts.yml      # On-prem servers
├── aws_ec2.yml           # AWS auto-discovery
├── azure_rm.yml          # Azure auto-discovery
└── group_vars/
    ├── all.yml
    └── env_production.yml

# Use entire directory as inventory
ansible-playbook -i inventory/ site.yml

Custom Inventory Script

#!/usr/bin/env python3
"""Custom dynamic inventory script for internal CMDB."""
import json
import sys
import requests

def get_inventory():
    response = requests.get("https://cmdb.internal/api/hosts")
    hosts = response.json()

    inventory = {
        "_meta": {"hostvars": {}},
        "all": {"children": ["webservers", "dbservers"]},
        "webservers": {"hosts": []},
        "dbservers": {"hosts": []},
    }

    for host in hosts:
        group = host["role"]
        if group in inventory:
            inventory[group]["hosts"].append(host["fqdn"])
        inventory["_meta"]["hostvars"][host["fqdn"]] = {
            "ansible_host": host["ip"],
            "ansible_user": host.get("ssh_user", "deploy"),
        }

    return inventory

if __name__ == "__main__":
    if "--list" in sys.argv:
        print(json.dumps(get_inventory(), indent=2))
    elif "--host" in sys.argv:
        print(json.dumps({}))

chmod +x inventory/custom_cmdb.py
ansible-inventory -i inventory/custom_cmdb.py --list

Caching

# ansible.cfg — cache inventory for faster runs
[inventory]
cache = true
cache_plugin = jsonfile
cache_connection = /tmp/ansible-inventory-cache
cache_timeout = 300  # 5 minutes

Troubleshooting

Issue	Solution
No hosts found	Check filters, verify credentials, test with `--list`
Authentication failed	Set env vars (`AWS_ACCESS_KEY_ID`) or check service account
Wrong hostname used	Adjust `hostnames` or `compose.ansible_host`
Slow inventory load	Enable caching, restrict regions/resource groups
Plugin not found	Run `ansible-galaxy collection install`, check `enable_plugins`
Groups not created	Verify `keyed_groups` key exists in instance metadata

Best Practices

Use keyed_groups for automatic grouping — tags become Ansible groups
Filter aggressively — only discover hosts you'll actually manage
Cache in CI/CD — avoid API calls on every task
Combine static + dynamic — use directory as inventory source
Set compose.ansible_host — ensure correct IP (public vs private)
Tag your instances — consistent tags = consistent inventory groups

Conclusion

Dynamic inventory eliminates manual host management. Configure an aws_ec2.yml, azure_rm.yml, or gcp.yml file with appropriate filters and keyed_groups. Instances are automatically grouped by tags, regions, and instance types. Combine with static inventory for hybrid environments. Cache results to avoid slow API calls during playbook execution.