Ansible AWX Backup and Restore — Protect Your Automation

Introduction

Ansible AWX stores job templates, inventories, credentials, schedules, and execution history in a PostgreSQL database. Losing this data means rebuilding your entire automation platform from scratch. This guide covers backup and restore procedures for both Kubernetes (AWX Operator) and Docker Compose deployments.

What Gets Backed Up

Component	Contains	Critical?
PostgreSQL database	All AWX data (jobs, templates, inventories, users)	✅ Essential
Secret key	Encryption key for credentials	✅ Essential
`/var/lib/awx/projects/`	Manual project files	If not in SCM
Custom EE images	Execution environment definitions	Rebuild from Containerfile
`settings.py` overrides	Custom configuration	✅ Important

Kubernetes (AWX Operator) Backup

Create a Backup

# awx-backup.yml
apiVersion: awx.ansible.com/v1beta1
kind: AWXBackup
metadata:
  name: awx-backup-2026-06-01
  namespace: awx
spec:
  deployment_name: awx
  backup_pvc: awx-backup-claim
  backup_pvc_namespace: awx
  # Optional: backup to specific storage class
  backup_storage_class: standard
  # Keep specific number of backups
  backup_resource_requirements:
    requests:
      storage: 10Gi

# Apply the backup resource
kubectl apply -f awx-backup.yml

# Monitor backup progress
kubectl get awxbackup -n awx -w
kubectl logs -n awx -l app.kubernetes.io/name=awx-operator -f

Automated Backup CronJob

# awx-backup-cronjob.yml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: awx-scheduled-backup
  namespace: awx
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: awx-operator-controller-manager
          containers:
          - name: backup-trigger
            image: bitnami/kubectl:latest
            command:
            - /bin/sh
            - -c
            - |
              TIMESTAMP=$(date +%Y%m%d-%H%M%S)
              cat <<YAML | kubectl apply -f -
              apiVersion: awx.ansible.com/v1beta1
              kind: AWXBackup
              metadata:
                name: awx-backup-${TIMESTAMP}
                namespace: awx
              spec:
                deployment_name: awx
                backup_pvc: awx-backup-claim
              YAML
          restartPolicy: OnFailure

Restore from Backup

# awx-restore.yml
apiVersion: awx.ansible.com/v1beta1
kind: AWXRestore
metadata:
  name: awx-restore
  namespace: awx
spec:
  deployment_name: awx
  backup_name: awx-backup-2026-06-01
  # Or specify PVC directly:
  # backup_pvc: awx-backup-claim
  # backup_dir: /backups/tower-openshift-backup-2026-06-01

# Apply restore
kubectl apply -f awx-restore.yml

# Monitor
kubectl get awxrestore -n awx -w

# Verify AWX is running after restore
kubectl get pods -n awx
kubectl port-forward svc/awx-service -n awx 8080:80

Docker Compose Backup

Manual Database Backup

# Dump the PostgreSQL database
docker exec awx_postgres pg_dump -U awx -F c awx > awx-backup-$(date +%Y%m%d).dump

# Backup the secret key
docker exec awx_task cat /etc/tower/SECRET_KEY > secret_key.bak

# Backup custom settings
docker cp awx_task:/etc/tower/settings.py ./settings.py.bak

# Backup project files (if using manual projects)
docker cp awx_task:/var/lib/awx/projects/ ./projects-backup/

Automated Backup Script

#!/bin/bash
# awx-backup.sh — Run daily via cron
set -euo pipefail

BACKUP_DIR="/opt/awx-backups"
RETENTION_DAYS=30
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
CURRENT_BACKUP="${BACKUP_DIR}/${TIMESTAMP}"

mkdir -p "${CURRENT_BACKUP}"

echo "=== AWX Backup Started: ${TIMESTAMP} ==="

# Database dump
docker exec awx_postgres pg_dump -U awx -F c awx \
  > "${CURRENT_BACKUP}/awx-db.dump"

# Secret key
docker exec awx_task cat /etc/tower/SECRET_KEY \
  > "${CURRENT_BACKUP}/SECRET_KEY"

# Settings
docker cp awx_task:/etc/tower/settings.py \
  "${CURRENT_BACKUP}/settings.py" 2>/dev/null || true

# Compress
tar -czf "${CURRENT_BACKUP}.tar.gz" -C "${BACKUP_DIR}" "${TIMESTAMP}"
rm -rf "${CURRENT_BACKUP}"

# Cleanup old backups
find "${BACKUP_DIR}" -name "*.tar.gz" -mtime +${RETENTION_DAYS} -delete

echo "=== Backup complete: ${CURRENT_BACKUP}.tar.gz ==="
echo "Size: $(du -h "${CURRENT_BACKUP}.tar.gz" | cut -f1)"

# Add to crontab
# crontab -e
0 3 * * * /opt/scripts/awx-backup.sh >> /var/log/awx-backup.log 2>&1

Restore (Docker Compose)

# Stop AWX services
docker compose stop awx_task awx_web

# Restore database
docker exec -i awx_postgres pg_restore -U awx -d awx --clean --if-exists \
  < /path/to/awx-db.dump

# Restore secret key
docker cp /path/to/SECRET_KEY awx_task:/etc/tower/SECRET_KEY

# Restart
docker compose up -d

Ansible Playbook for AWX Backup

---
- name: Backup AWX installation
  hosts: awx_server
  become: true
  vars:
    backup_dir: /opt/awx-backups
    retention_days: 30
    remote_backup_host: backup.example.com
    remote_backup_path: /backups/awx/

  tasks:
    - name: Create backup directory
      ansible.builtin.file:
        path: "{{ backup_dir }}"
        state: directory
        mode: "0700"

    - name: Dump PostgreSQL database
      community.docker.docker_container_exec:
        container: awx_postgres
        command: pg_dump -U awx -F c awx
      register: db_dump

    - name: Save database dump
      ansible.builtin.copy:
        content: "{{ db_dump.stdout }}"
        dest: "{{ backup_dir }}/awx-db-{{ ansible_date_time.date }}.dump"
        mode: "0600"

    - name: Extract secret key
      community.docker.docker_container_exec:
        container: awx_task
        command: cat /etc/tower/SECRET_KEY
      register: secret_key

    - name: Save secret key
      ansible.builtin.copy:
        content: "{{ secret_key.stdout }}"
        dest: "{{ backup_dir }}/SECRET_KEY"
        mode: "0600"

    - name: Compress backup
      community.general.archive:
        path: "{{ backup_dir }}"
        dest: "/tmp/awx-backup-{{ ansible_date_time.date }}.tar.gz"
        format: gz

    - name: Copy to remote backup server
      ansible.builtin.synchronize:
        src: "/tmp/awx-backup-{{ ansible_date_time.date }}.tar.gz"
        dest: "{{ remote_backup_path }}"
        mode: push
      delegate_to: "{{ inventory_hostname }}"

    - name: Cleanup old local backups
      ansible.builtin.find:
        paths: "{{ backup_dir }}"
        age: "{{ retention_days }}d"
      register: old_backups

    - name: Remove old backups
      ansible.builtin.file:
        path: "{{ item.path }}"
        state: absent
      loop: "{{ old_backups.files }}"

Verify Backup Integrity

# Test restore to a temporary database
docker exec awx_postgres createdb -U awx awx_test
docker exec -i awx_postgres pg_restore -U awx -d awx_test < awx-db.dump

# Check data
docker exec awx_postgres psql -U awx -d awx_test -c \
  "SELECT count(*) FROM main_jobtemplate;"

# Cleanup
docker exec awx_postgres dropdb -U awx awx_test

Troubleshooting

Issue	Solution
Backup PVC full	Increase storage or reduce retention
Restore fails with version mismatch	AWX version must match between backup and restore
Secret key lost	Credentials stored in AWX become unrecoverable — always back up SECRET_KEY
Operator backup stuck	Check operator logs: `kubectl logs -n awx deployment/awx-operator-controller-manager`
Permission denied on PVC	Check RBAC and PVC access modes
pg_restore errors	Use `--clean --if-exists` flags; ensure target DB exists

Best Practices

Always back up SECRET_KEY — without it, all stored credentials are lost
Test restores regularly — untested backups are not backups
Store backups off-cluster — S3, NFS, or remote server
Version-match on restore — back up before AWX upgrades
Encrypt backup files — they contain sensitive credential data
Monitor backup jobs — alert on failure

Conclusion

AWX backup and restore protects your automation investment. Use the AWX Operator's built-in AWXBackup/AWXRestore resources on Kubernetes, or pg_dump/pg_restore for Docker Compose. Automate daily backups, store them off-site, and test restores quarterly. The SECRET_KEY is the single most critical file — lose it and all encrypted credentials become unrecoverable.