Ansible Lookup Plugins — Query External Data Sources

Introduction

Lookup plugins fetch data from outside Ansible — files, environment variables, command output, URLs, password stores, and more. They run on the controller (not remote hosts) and return data that you can use in variables, templates, and task parameters. This guide covers the most useful built-in lookups with practical examples.

Syntax

# Jinja2 lookup function
"{{ lookup('plugin_name', 'argument') }}"

# With options
"{{ lookup('plugin_name', 'argument', option='value') }}"

# Query (returns list, fails on missing)
"{{ query('plugin_name', 'argument') }}"

# q() is shorthand for query()
"{{ q('file', '/etc/hostname') }}"

file — Read File Contents

    - name: Read SSH public key
      ansible.builtin.authorized_key:
        user: deploy
        key: "{{ lookup('file', '~/.ssh/id_ed25519.pub') }}"

    - name: Deploy certificate from local file
      ansible.builtin.copy:
        content: "{{ lookup('file', 'certs/app.pem') }}"
        dest: /etc/ssl/certs/app.pem
        mode: '0644'

    - name: Read multiple files
      ansible.builtin.debug:
        msg: "{{ lookup('file', item) }}"
      loop:
        - /etc/hostname
        - /etc/os-release

env — Environment Variables

    - name: Use controller's environment
      ansible.builtin.debug:
        msg: "Home: {{ lookup('env', 'HOME') }}"

    - name: AWS credentials from environment
      amazon.aws.ec2_instance:
        name: webserver
        instance_type: t3.micro
      environment:
        AWS_ACCESS_KEY_ID: "{{ lookup('env', 'AWS_ACCESS_KEY_ID') }}"
        AWS_SECRET_ACCESS_KEY: "{{ lookup('env', 'AWS_SECRET_ACCESS_KEY') }}"

    - name: Conditional based on environment
      ansible.builtin.debug:
        msg: "Running in CI"
      when: lookup('env', 'CI') | default('') | length > 0

pipe — Command Output

    - name: Get Git commit hash
      ansible.builtin.set_fact:
        git_hash: "{{ lookup('pipe', 'git rev-parse --short HEAD') }}"

    - name: Get current date
      ansible.builtin.set_fact:
        build_date: "{{ lookup('pipe', 'date +%Y-%m-%d') }}"

    - name: Generate random password
      ansible.builtin.set_fact:
        temp_password: "{{ lookup('pipe', 'openssl rand -base64 24') }}"
      no_log: true

url — Fetch from URLs

    - name: Get latest release version
      ansible.builtin.set_fact:
        latest_version: "{{ lookup('url', 'https://api.github.com/repos/org/app/releases/latest') | from_json | json_query('tag_name') }}"

    - name: Download configuration
      ansible.builtin.copy:
        content: "{{ lookup('url', 'https://config.example.com/app.conf', headers={'Authorization': 'Bearer ' + api_token}) }}"
        dest: /etc/myapp/app.conf

password — Generate and Store Passwords

    - name: Generate password (stored in file)
      ansible.builtin.user:
        name: deploy
        password: "{{ lookup('password', '/tmp/deploy-password chars=ascii_letters,digits length=24') | password_hash('sha512') }}"
      no_log: true

    - name: Generate password (not stored)
      ansible.builtin.set_fact:
        db_password: "{{ lookup('password', '/dev/null chars=ascii_letters,digits,punctuation length=32') }}"
      no_log: true

template — Render Jinja2 Templates

    - name: Render template as variable
      ansible.builtin.set_fact:
        rendered_config: "{{ lookup('template', 'templates/config.yml.j2') }}"

    - name: Use rendered template in API call
      ansible.builtin.uri:
        url: "https://api.example.com/config"
        method: PUT
        body: "{{ lookup('template', 'templates/api-payload.json.j2') }}"
        body_format: json

csvfile — Read CSV Data

    # users.csv:
    # username,role,department
    # alice,admin,engineering
    # bob,user,marketing

    - name: Get user's role from CSV
      ansible.builtin.debug:
        msg: "Alice's role: {{ lookup('csvfile', 'alice file=users.csv delimiter=, col=1') }}"

    - name: Get department
      ansible.builtin.debug:
        msg: "Bob's dept: {{ lookup('csvfile', 'bob file=users.csv delimiter=, col=2') }}"

ini — Read INI Files

    # app.ini:
    # [database]
    # host = db.example.com
    # port = 5432

    - name: Read INI value
      ansible.builtin.debug:
        msg: "DB host: {{ lookup('ini', 'host section=database file=app.ini') }}"

sequence — Generate Number Sequences

    - name: Create numbered directories
      ansible.builtin.file:
        path: "/opt/app/worker-{{ item }}"
        state: directory
      loop: "{{ query('sequence', 'start=1 end=10') }}"

    - name: Generate IP range
      ansible.builtin.debug:
        msg: "{{ item }}"
      loop: "{{ query('sequence', 'start=10 end=20 format=192.168.1.%d') }}"

together and zip — Combine Lists

    - name: Create users with specific UIDs
      ansible.builtin.user:
        name: "{{ item.0 }}"
        uid: "{{ item.1 }}"
      loop: "{{ query('together', users, uids) }}"
      vars:
        users: [alice, bob, charlie]
        uids: [1001, 1002, 1003]

lookup vs query

    # lookup — returns comma-separated string (legacy)
    "{{ lookup('file', 'a.txt', 'b.txt') }}"
    # Returns: "content_a,content_b"

    # query — returns list (preferred)
    "{{ query('file', 'a.txt', 'b.txt') }}"
    # Returns: ["content_a", "content_b"]

    # query fails on missing; lookup returns empty
    # Use wantlist=True with lookup to get list behavior
    "{{ lookup('file', 'a.txt', wantlist=True) }}"

Error Handling

    # Default value on failure
    - name: Read optional config
      ansible.builtin.set_fact:
        optional_config: "{{ lookup('file', '/etc/myapp/optional.conf', errors='ignore') | default('') }}"

    # Fail explicitly
    - name: Read required config
      ansible.builtin.set_fact:
        required_config: "{{ lookup('file', '/etc/myapp/required.conf', errors='strict') }}"

Troubleshooting

Issue	Solution
"file not found"	Lookup runs on controller; path is relative to playbook
`env` returns empty	Variable not set on controller machine
`pipe` fails silently	Check command works manually on controller
`url` timeout	Add `timeout=30` parameter
Wrong data type	Use `query()` for lists, `lookup()` for strings

Best Practices

Use query() over lookup() — returns lists, fails on errors
Use env sparingly — prefer Vault for secrets over environment variables
Cache expensive lookups — set_fact the result instead of calling repeatedly
Use errors='ignore' for optional data — with | default() fallback
Lookups run on controller — never use for remote file access
no_log: true on password/secret lookups

Conclusion

Lookup plugins bridge Ansible with external data — files, environment, commands, URLs, and more. They run on the controller and inject data into your playbooks at runtime. Master file, env, pipe, and password for daily use; reach for url, csvfile, and template for advanced data integration. Always prefer query() over lookup() for cleaner, more predictable behavior.