Ansible for Agentic AI — Automate Multi-Agent Infrastructure

Introduction

Agentic AI — autonomous AI systems that plan, execute, and iterate on tasks — is the defining tech trend of 2026. Multi-agent architectures require complex infrastructure: GPU clusters, vector databases, message queues, API gateways, and monitoring. Ansible automates the provisioning, deployment, and lifecycle management of this entire stack.

AI Agent Infrastructure Stack

A typical multi-agent system needs:

Component	Tools	Ansible Role
LLM Inference	vLLM, TGI, Ollama	Deploy + configure
Agent Framework	LangChain, CrewAI, AutoGen	Install + configure
Vector Database	Qdrant, Weaviate, Milvus	Deploy + scale
Message Queue	Redis, RabbitMQ, NATS	Cluster setup
API Gateway	Kong, Traefik	Route + rate-limit
Monitoring	Prometheus, Grafana, LangSmith	Observe agents
GPU Management	NVIDIA drivers, CUDA	Provision + maintain

Deploy an LLM Inference Server

vLLM (High-Performance)

---
- name: Deploy vLLM inference server
  hosts: gpu_servers
  become: true
  vars:
    model_name: "meta-llama/Llama-3.1-70B-Instruct"
    vllm_port: 8000
    gpu_memory_utilization: 0.9
    max_model_len: 8192

  tasks:
    - name: Install NVIDIA Container Toolkit
      ansible.builtin.include_role:
        name: nvidia_container_toolkit

    - name: Create vLLM directories
      ansible.builtin.file:
        path: "{{ item }}"
        state: directory
        mode: "0755"
      loop:
        - /opt/vllm
        - /opt/vllm/models
        - /opt/vllm/logs

    - name: Deploy vLLM container
      community.docker.docker_container:
        name: vllm-inference
        image: vllm/vllm-openai:latest
        state: started
        restart_policy: unless-stopped
        ports:
          - "{{ vllm_port }}:8000"
        volumes:
          - /opt/vllm/models:/root/.cache/huggingface
        env:
          HUGGING_FACE_HUB_TOKEN: "{{ vault_hf_token }}"
        command: >
          --model {{ model_name }}
          --gpu-memory-utilization {{ gpu_memory_utilization }}
          --max-model-len {{ max_model_len }}
          --tensor-parallel-size {{ ansible_local_gpus | default(1) }}
        device_requests:
          - driver: nvidia
            count: -1
            capabilities:
              - - gpu
      no_log: true

    - name: Wait for vLLM to load model
      ansible.builtin.uri:
        url: "http://localhost:{{ vllm_port }}/health"
        status_code: 200
      register: health
      until: health.status == 200
      retries: 60
      delay: 10

    - name: Verify inference
      ansible.builtin.uri:
        url: "http://localhost:{{ vllm_port }}/v1/models"
      register: models
    
    - name: Show loaded models
      ansible.builtin.debug:
        msg: "Models: {{ models.json.data | map(attribute='id') | list }}"

Ollama (Local Development)

- name: Deploy Ollama for local agents
  hosts: dev_machines
  become: true
  tasks:
    - name: Install Ollama
      ansible.builtin.shell: curl -fsSL https://ollama.com/install.sh | sh
      args:
        creates: /usr/local/bin/ollama

    - name: Pull models for agents
      ansible.builtin.command: "ollama pull {{ item }}"
      loop:
        - llama3.1:8b
        - nomic-embed-text
        - codellama:13b
      register: pull_result
      changed_when: "'pulling' in pull_result.stdout"

Deploy Vector Database (Agent Memory)

- name: Deploy Qdrant vector database
  hosts: vector_db
  become: true
  vars:
    qdrant_port: 6333
    qdrant_grpc_port: 6334

  tasks:
    - name: Create Qdrant data directory
      ansible.builtin.file:
        path: /opt/qdrant/data
        state: directory
        mode: "0755"

    - name: Deploy Qdrant
      community.docker.docker_container:
        name: qdrant
        image: qdrant/qdrant:latest
        state: started
        restart_policy: unless-stopped
        ports:
          - "{{ qdrant_port }}:6333"
          - "{{ qdrant_grpc_port }}:6334"
        volumes:
          - /opt/qdrant/data:/qdrant/storage
        env:
          QDRANT__SERVICE__API_KEY: "{{ vault_qdrant_api_key }}"
      no_log: true

    - name: Create agent memory collection
      ansible.builtin.uri:
        url: "http://localhost:{{ qdrant_port }}/collections/agent_memory"
        method: PUT
        body_format: json
        body:
          vectors:
            size: 1536
            distance: Cosine
          optimizers_config:
            indexing_threshold: 10000
        headers:
          api-key: "{{ vault_qdrant_api_key }}"
        status_code: [200, 409]
      no_log: true

Deploy Multi-Agent Framework

CrewAI Application

- name: Deploy CrewAI multi-agent system
  hosts: agent_servers
  become: true
  vars:
    app_dir: /opt/crewai-app
    app_user: crewai

  tasks:
    - name: Create application user
      ansible.builtin.user:
        name: "{{ app_user }}"
        system: true
        shell: /usr/sbin/nologin

    - name: Create app directory
      ansible.builtin.file:
        path: "{{ app_dir }}"
        state: directory
        owner: "{{ app_user }}"

    - name: Deploy application code
      ansible.builtin.git:
        repo: "{{ crewai_repo }}"
        dest: "{{ app_dir }}"
        version: "{{ app_version }}"
      become_user: "{{ app_user }}"
      notify: Restart CrewAI

    - name: Install Python dependencies
      ansible.builtin.pip:
        requirements: "{{ app_dir }}/requirements.txt"
        virtualenv: "{{ app_dir }}/venv"
      become_user: "{{ app_user }}"

    - name: Configure agent environment
      ansible.builtin.template:
        src: crewai-env.j2
        dest: "{{ app_dir }}/.env"
        owner: "{{ app_user }}"
        mode: "0600"
      no_log: true
      notify: Restart CrewAI

    - name: Create systemd service
      ansible.builtin.template:
        src: crewai.service.j2
        dest: /etc/systemd/system/crewai.service
      notify:
        - Reload systemd
        - Restart CrewAI

    - name: Start CrewAI service
      ansible.builtin.service:
        name: crewai
        state: started
        enabled: true

  handlers:
    - name: Reload systemd
      ansible.builtin.systemd:
        daemon_reload: true

    - name: Restart CrewAI
      ansible.builtin.service:
        name: crewai
        state: restarted

Environment template:

{# templates/crewai-env.j2 #}
# LLM Configuration
OPENAI_API_KEY={{ vault_openai_key }}
OPENAI_MODEL_NAME={{ llm_model | default('gpt-4o') }}

# Local LLM (alternative)
{% if use_local_llm | default(false) %}
OPENAI_API_BASE=http://{{ hostvars[groups['gpu_servers'][0]]['ansible_host'] }}:8000/v1
{% endif %}

# Vector DB
QDRANT_URL=http://{{ hostvars[groups['vector_db'][0]]['ansible_host'] }}:6333
QDRANT_API_KEY={{ vault_qdrant_api_key }}

# Agent Config
MAX_ITERATIONS={{ max_agent_iterations | default(10) }}
VERBOSE={{ agent_verbose | default('false') }}

GPU Cluster Provisioning

- name: Provision GPU cluster for AI agents
  hosts: gpu_servers
  become: true
  tasks:
    - name: Install NVIDIA driver
      ansible.builtin.package:
        name: nvidia-driver-560
        state: present
      notify: Reboot server

    - name: Install CUDA toolkit
      ansible.builtin.package:
        name: cuda-toolkit-12-6
        state: present

    - name: Verify GPU availability
      ansible.builtin.command: nvidia-smi --query-gpu=name,memory.total --format=csv,noheader
      register: gpu_info
      changed_when: false

    - name: Display GPU inventory
      ansible.builtin.debug:
        msg: "{{ inventory_hostname }}: {{ gpu_info.stdout_lines }}"

    - name: Set GPU facts
      ansible.builtin.set_fact:
        gpu_count: "{{ gpu_info.stdout_lines | length }}"
        cacheable: true

  handlers:
    - name: Reboot server
      ansible.builtin.reboot:
        reboot_timeout: 600

Agent Monitoring Stack

- name: Deploy agent monitoring
  hosts: monitoring
  become: true
  tasks:
    - name: Deploy Prometheus for agent metrics
      community.docker.docker_container:
        name: prometheus
        image: prom/prometheus:latest
        ports: ["9090:9090"]
        volumes:
          - /opt/monitoring/prometheus.yml:/etc/prometheus/prometheus.yml

    - name: Deploy Grafana dashboards
      community.docker.docker_container:
        name: grafana
        image: grafana/grafana:latest
        ports: ["3000:3000"]
        env:
          GF_SECURITY_ADMIN_PASSWORD: "{{ vault_grafana_pass }}"
      no_log: true

    - name: Import AI agent dashboard
      ansible.builtin.uri:
        url: http://localhost:3000/api/dashboards/db
        method: POST
        headers:
          Authorization: "Bearer {{ vault_grafana_api_key }}"
        body_format: json
        body: "{{ lookup('file', 'dashboards/ai-agents.json') }}"
      no_log: true

Complete Multi-Agent Playbook

---
# site.yml — Full agentic AI infrastructure
- import_playbook: playbooks/gpu-cluster.yml
- import_playbook: playbooks/vector-db.yml
- import_playbook: playbooks/llm-inference.yml
- import_playbook: playbooks/agent-framework.yml
- import_playbook: playbooks/monitoring.yml
- import_playbook: playbooks/api-gateway.yml

# Deploy entire stack
ansible-playbook site.yml -e "env=production app_version=v2.1.0"

# Scale inference servers
ansible-playbook playbooks/llm-inference.yml --limit gpu_servers_new

# Update agent code only
ansible-playbook playbooks/agent-framework.yml -e "app_version=v2.2.0" --tags deploy

Conclusion

Ansible is the infrastructure backbone for agentic AI. It provisions GPU clusters, deploys LLM inference servers (vLLM, Ollama), manages vector databases (Qdrant, Weaviate) for agent memory, and orchestrates multi-agent frameworks (CrewAI, LangChain). The key pattern: treat each AI component as a deployable service, use Ansible roles for each layer, and manage the full stack with a single site.yml. As multi-agent systems scale from pilots to production in 2026, infrastructure automation isn't optional — it's the foundation.