Ansible for AI Infrastructure — GPU Clusters, Model

Introduction

2026's AI infrastructure challenge isn't just buying GPUs — it's balancing model choice, inference cost, deployment architecture, and token economics. Deloitte calls this an "AI infrastructure reckoning." Ansible automates GPU cluster lifecycle, model serving deployment, multi-model routing, cost optimization, and capacity planning.

GPU Cluster Provisioning

---
- name: Provision GPU cluster
  hosts: gpu_nodes
  become: true
  tasks:
    - name: Install NVIDIA driver
      ansible.builtin.package:
        name: "nvidia-driver-{{ nvidia_driver_version | default('560') }}"
        state: present
      notify: Reboot for driver

    - name: Install CUDA toolkit
      ansible.builtin.package:
        name: "cuda-toolkit-{{ cuda_version | default('12-6') }}"
        state: present

    - name: Install cuDNN
      ansible.builtin.package:
        name: libcudnn9-cuda-12
        state: present

    - name: Install NCCL (multi-GPU communication)
      ansible.builtin.package:
        name: libnccl2
        state: present

    - name: Verify GPU inventory
      ansible.builtin.command: >
        nvidia-smi --query-gpu=index,name,memory.total,driver_version
        --format=csv,noheader
      register: gpu_inventory
      changed_when: false

    - name: Report GPU fleet status
      ansible.builtin.debug:
        msg: |
          Host: {{ inventory_hostname }}
          GPUs:
          {{ gpu_inventory.stdout }}

    - name: Set GPU persistence mode
      ansible.builtin.command: nvidia-smi -pm 1
      changed_when: false

    - name: Configure GPU clock speeds for inference
      ansible.builtin.command: >
        nvidia-smi -lgc {{ gpu_min_clock }},{{ gpu_max_clock }}
      when: gpu_clock_lock | default(false)
      changed_when: false

  handlers:
    - name: Reboot for driver
      ansible.builtin.reboot:
        reboot_timeout: 600

NVIDIA Triton Inference Server

- name: Deploy Triton Inference Server
  hosts: inference_servers
  become: true
  vars:
    triton_version: "24.09"
    model_repo: /opt/triton/models

  tasks:
    - name: Create model repository
      ansible.builtin.file:
        path: "{{ model_repo }}/{{ item }}/1"
        state: directory
      loop: "{{ triton_models }}"

    - name: Deploy model configurations
      ansible.builtin.template:
        src: "triton/{{ item }}/config.pbtxt.j2"
        dest: "{{ model_repo }}/{{ item }}/config.pbtxt"
      loop: "{{ triton_models }}"

    - name: Deploy Triton container
      community.docker.docker_container:
        name: triton
        image: "nvcr.io/nvidia/tritonserver:{{ triton_version }}-py3"
        state: started
        restart_policy: unless-stopped
        ports:
          - "8000:8000"   # HTTP
          - "8001:8001"   # gRPC
          - "8002:8002"   # Metrics
        volumes:
          - "{{ model_repo }}:/models"
        command: "tritonserver --model-repository=/models --log-verbose=0"
        device_requests:
          - driver: nvidia
            count: -1
            capabilities: [[gpu]]

    - name: Wait for Triton ready
      ansible.builtin.uri:
        url: http://localhost:8000/v2/health/ready
      register: triton_health
      until: triton_health.status == 200
      retries: 30
      delay: 5

    - name: Verify loaded models
      ansible.builtin.uri:
        url: http://localhost:8000/v2/models
      register: loaded_models

    - name: Display model status
      ansible.builtin.debug:
        msg: "Loaded: {{ loaded_models.json.models | map(attribute='name') | list }}"

Multi-Model Routing (Cost Optimization)

- name: Deploy model router for cost optimization
  hosts: api_gateway
  become: true
  tasks:
    - name: Deploy model router configuration
      ansible.builtin.template:
        src: model-router.yml.j2
        dest: /opt/model-router/config.yml
      notify: Restart model router

    - name: Deploy model router service
      community.docker.docker_container:
        name: model-router
        image: "{{ model_router_image }}"
        state: started
        ports: ["8080:8080"]
        volumes:
          - /opt/model-router/config.yml:/app/config.yml

{# templates/model-router.yml.j2 #}
# Route requests to cheapest adequate model
models:
  - name: gpt-4o
    endpoint: https://api.openai.com/v1
    cost_per_1k_tokens: 0.005
    quality_tier: premium
    max_tokens: 128000

  - name: llama-3.1-70b
    endpoint: http://{{ hostvars[groups['inference_servers'][0]]['ansible_host'] }}:8000/v1
    cost_per_1k_tokens: 0.0002
    quality_tier: standard
    max_tokens: 8192

  - name: llama-3.1-8b
    endpoint: http://{{ hostvars[groups['inference_servers'][1]]['ansible_host'] }}:8000/v1
    cost_per_1k_tokens: 0.00005
    quality_tier: economy
    max_tokens: 8192

routing_rules:
  - match: { complexity: high, quality: premium }
    model: gpt-4o
  - match: { complexity: medium }
    model: llama-3.1-70b
  - match: { complexity: low }
    model: llama-3.1-8b
  - default: llama-3.1-70b

budget:
  daily_limit_usd: {{ ai_daily_budget | default(100) }}
  alert_threshold: 0.8

GPU Monitoring and Autoscaling

- name: Deploy GPU monitoring
  hosts: gpu_nodes
  become: true
  tasks:
    - name: Deploy DCGM exporter
      community.docker.docker_container:
        name: dcgm-exporter
        image: nvcr.io/nvidia/k8s/dcgm-exporter:latest
        state: started
        ports: ["9400:9400"]
        device_requests:
          - driver: nvidia
            count: -1
            capabilities: [[gpu]]

    - name: Deploy GPU utilization alerting
      ansible.builtin.template:
        src: gpu-alerts.yml.j2
        dest: /etc/prometheus/rules/gpu-alerts.yml
      notify: Reload Prometheus
      delegate_to: "{{ monitoring_host }}"

{# templates/gpu-alerts.yml.j2 #}
groups:
  - name: gpu_alerts
    rules:
      - alert: GPUHighUtilization
        expr: DCGM_FI_DEV_GPU_UTIL > 95
        for: 10m
        labels: { severity: warning }
        annotations:
          summary: "GPU {{ $labels.gpu }} on {{ $labels.instance }} above 95%"

      - alert: GPUMemoryExhausted
        expr: DCGM_FI_DEV_FB_USED / DCGM_FI_DEV_FB_FREE > 0.95
        for: 5m
        labels: { severity: critical }

      - alert: GPUTemperature
        expr: DCGM_FI_DEV_GPU_TEMP > 85
        for: 5m
        labels: { severity: critical }

      - alert: InferenceLatencyHigh
        expr: histogram_quantile(0.99, rate(inference_duration_seconds_bucket[5m])) > 2
        for: 5m
        labels: { severity: warning }

Capacity Planning

- name: GPU capacity audit
  hosts: gpu_nodes
  become: true
  tasks:
    - name: Collect GPU metrics
      ansible.builtin.command: >
        nvidia-smi --query-gpu=name,memory.total,memory.used,utilization.gpu,temperature.gpu,power.draw
        --format=csv,noheader,nounits
      register: gpu_metrics
      changed_when: false

    - name: Calculate fleet capacity
      ansible.builtin.set_fact:
        gpu_data: >-
          {{ gpu_data | default([]) + [{
            'host': inventory_hostname,
            'gpu': item.split(',')[0] | trim,
            'mem_total_mb': item.split(',')[1] | trim | int,
            'mem_used_mb': item.split(',')[2] | trim | int,
            'util_pct': item.split(',')[3] | trim | int,
            'temp_c': item.split(',')[4] | trim | int,
            'power_w': item.split(',')[5] | trim | float
          }] }}
      loop: "{{ gpu_metrics.stdout_lines }}"

    - name: Generate capacity report
      ansible.builtin.template:
        src: gpu-capacity.j2
        dest: /tmp/gpu-capacity-report.json
      delegate_to: localhost
      run_once: true

Conclusion

AI infrastructure in 2026 is about optimization, not just procurement. Ansible automates the full stack: GPU provisioning with driver pinning and clock management, Triton/vLLM model serving, multi-model routing for cost optimization, DCGM monitoring with utilization alerts, and fleet capacity planning. The key insight: treat inference infrastructure like any other fleet — provision it declaratively, monitor it continuously, and optimize costs by routing to the cheapest adequate model. Ansible makes the "AI infrastructure reckoning" manageable at scale.