Ansible Domain-Specific Language Models Fine-Tuning

Introduction

General-purpose LLMs are powerful but often insufficient for specialized domains — legal, medical, financial, scientific. Domain-specific language models (DSLMs) are fine-tuned on industry data to deliver superior accuracy in narrow fields. Building and serving these models requires substantial infrastructure: GPU clusters for training, data pipelines for curation, model registries for versioning, and inference servers for production. Ansible automates the full lifecycle.

Domain-Specific LM Architecture

┌──────────────────────────────────────────────────┐
│          Domain-Specific LM Platform              │
│                                                    │
│  ┌──────────┐  ┌──────────┐  ┌───────────────┐  │
│  │ Training │  │ Data     │  │ Model         │  │
│  │ Cluster  │  │ Pipeline │  │ Registry      │  │
│  │ (GPU)    │  │          │  │ (MLflow)      │  │
│  └──────────┘  └──────────┘  └───────────────┘  │
│                                                    │
│  ┌──────────┐  ┌──────────┐  ┌───────────────┐  │
│  │ Eval     │  │ Serving  │  │ Monitoring    │  │
│  │ Pipeline │  │ (vLLM)   │  │ & Drift       │  │
│  └──────────┘  └──────────┘  └───────────────┘  │
└──────────────────────────────────────────────────┘

Training Data Pipeline

# data-pipeline.yml
---
- name: Deploy training data pipeline
  hosts: data_servers
  become: true

  tasks:
    - name: Deploy data curation service
      community.docker.docker_container:
        name: data-curator
        image: "{{ data_curator_image }}"
        state: started
        volumes:
          - /data/raw:/data/raw:ro
          - /data/curated:/data/curated
        env:
          DEDUP_THRESHOLD: "0.85"
          PII_REMOVAL: "true"
          QUALITY_FILTER: "true"
          MIN_QUALITY_SCORE: "0.7"
          OUTPUT_FORMAT: "jsonl"

    - name: Configure data quality checks
      ansible.builtin.template:
        src: data-quality.yml.j2
        dest: /opt/pipeline/data-quality.yml
      vars:
        quality_rules:
          min_length: 50
          max_length: 32768
          remove_duplicates: true
          dedup_method: "minhash"
          remove_pii: true
          pii_patterns:
            - email
            - phone
            - ssn
            - credit_card
          language_filter: "{{ target_languages | default(['en']) }}"
          domain_relevance_threshold: 0.8

    - name: Schedule data processing
      ansible.builtin.cron:
        name: "Process new training data"
        hour: "2"
        minute: "0"
        job: "/opt/pipeline/process-data.sh >> /var/log/data-pipeline.log 2>&1"

Fine-Tuning Infrastructure

# fine-tuning-cluster.yml
---
- name: Provision fine-tuning GPU cluster
  hosts: training_nodes
  become: true

  vars:
    base_model: "meta-llama/Llama-3.1-70B"
    training_method: "qlora"  # qlora, lora, full
    gpu_type: "A100"
    gpus_per_node: 8

  tasks:
    - name: Install training dependencies
      ansible.builtin.pip:
        name:
          - torch==2.5.0
          - transformers==4.48.0
          - peft==0.14.0
          - trl==0.13.0
          - bitsandbytes==0.45.0
          - datasets==3.2.0
          - wandb
          - deepspeed
        virtualenv: /opt/training/venv

    - name: Deploy training configuration
      ansible.builtin.template:
        src: training-config.yml.j2
        dest: /opt/training/config.yml
      vars:
        training_config:
          model: "{{ base_model }}"
          method: "{{ training_method }}"
          dataset: "/data/curated/{{ domain }}"
          output_dir: "/models/{{ domain }}-{{ training_run_id }}"
          hyperparameters:
            learning_rate: 2.0e-5
            num_epochs: 3
            batch_size: 4
            gradient_accumulation_steps: 8
            warmup_ratio: 0.1
            weight_decay: 0.01
            max_seq_length: 4096
          lora_config:
            r: 64
            lora_alpha: 128
            target_modules: ["q_proj", "k_proj", "v_proj", "o_proj"]
            lora_dropout: 0.05
          deepspeed:
            stage: 3
            offload_optimizer: true
            offload_param: false

    - name: Deploy multi-node training launcher
      ansible.builtin.template:
        src: launch-training.sh.j2
        dest: /opt/training/launch.sh
        mode: "0755"
      vars:
        master_addr: "{{ hostvars[groups['training_nodes'][0]]['ansible_host'] }}"
        master_port: 29500
        world_size: "{{ groups['training_nodes'] | length * gpus_per_node }}"

    - name: Configure Weights & Biases tracking
      ansible.builtin.template:
        src: wandb-config.yml.j2
        dest: /opt/training/wandb.yml
      vars:
        wandb_project: "{{ domain }}-dslm"
        wandb_entity: "{{ org_name }}"
        wandb_tags: ["{{ domain }}", "{{ training_method }}", "{{ base_model | basename }}"]

Evaluation Pipeline

# eval-pipeline.yml
---
- name: Deploy model evaluation pipeline
  hosts: eval_servers
  become: true

  tasks:
    - name: Deploy evaluation service
      ansible.builtin.template:
        src: eval-config.yml.j2
        dest: /opt/eval/config.yml
      vars:
        eval_config:
          benchmarks:
            - name: "{{ domain }}_accuracy"
              dataset: "/data/eval/{{ domain }}_test.jsonl"
              metric: "accuracy"
              min_threshold: 0.85

            - name: "{{ domain }}_f1"
              dataset: "/data/eval/{{ domain }}_test.jsonl"
              metric: "f1_score"
              min_threshold: 0.80

            - name: "hallucination_rate"
              dataset: "/data/eval/{{ domain }}_factcheck.jsonl"
              metric: "factual_accuracy"
              min_threshold: 0.95

            - name: "general_capability"
              dataset: "mmlu"
              metric: "accuracy"
              min_threshold: 0.70
              description: "Ensure domain fine-tuning doesn't catastrophically degrade general ability"

          comparison:
            baseline_model: "{{ base_model }}"
            improvement_threshold: 0.05

    - name: Deploy automated eval runner
      ansible.builtin.cron:
        name: "Evaluate new model checkpoints"
        minute: "*/30"
        job: "/opt/eval/run-eval.sh >> /var/log/eval.log 2>&1"

Model Serving

# model-serving.yml
---
- name: Deploy domain-specific model serving
  hosts: inference_servers
  become: true

  tasks:
    - name: Deploy vLLM for domain model
      community.docker.docker_container:
        name: "dslm-{{ domain }}"
        image: vllm/vllm-openai:latest
        state: started
        restart_policy: unless-stopped
        ports:
          - "{{ serving_port }}:8000"
        env:
          MODEL_NAME: "/models/{{ domain }}-production"
          MAX_MODEL_LEN: "{{ max_context | default('8192') }}"
          GPU_MEMORY_UTILIZATION: "0.90"
          TENSOR_PARALLEL_SIZE: "{{ tensor_parallel | default('1') }}"
        device_requests:
          - driver: nvidia
            count: "{{ serving_gpus | default(1) }}"
            capabilities: [["gpu"]]
        volumes:
          - "/models:/models:ro"

    - name: Deploy model router (A/B testing)
      ansible.builtin.template:
        src: model-router.yml.j2
        dest: /opt/serving/router.yml
      vars:
        routes:
          - model: "{{ domain }}-v2"
            weight: 90
            endpoint: "http://localhost:{{ serving_port }}"
          - model: "{{ domain }}-v3-canary"
            weight: 10
            endpoint: "http://localhost:{{ canary_port }}"

    - name: Configure model monitoring
      ansible.builtin.template:
        src: model-monitor.yml.j2
        dest: /opt/monitoring/model-monitor.yml
      vars:
        monitoring:
          metrics:
            - latency_p99
            - tokens_per_second
            - error_rate
            - drift_score
          alerts:
            latency_p99_ms: 500
            error_rate_threshold: 0.01
            drift_score_threshold: 0.15

Industry Examples

# Legal domain LM
domain: legal
base_model: "meta-llama/Llama-3.1-70B"
training_data:
  - court_opinions
  - legal_briefs
  - regulatory_filings
  - contract_clauses
eval_benchmarks: [legal_qa, contract_analysis, case_citation]

# Medical domain LM
domain: medical
base_model: "meta-llama/Llama-3.1-70B"
training_data:
  - pubmed_abstracts
  - clinical_notes_deidentified
  - drug_interactions
  - medical_guidelines
eval_benchmarks: [medqa, clinical_reasoning, drug_safety]

# Financial domain LM
domain: finance
base_model: "meta-llama/Llama-3.1-70B"
training_data:
  - sec_filings
  - earnings_transcripts
  - financial_reports
  - market_analysis
eval_benchmarks: [finqa, sentiment_analysis, risk_assessment]

Conclusion

Domain-specific language models outperform general LLMs in specialized fields but require significant infrastructure: data pipelines for curation and quality control, GPU clusters for fine-tuning, evaluation pipelines for benchmarking, and inference servers for production serving. Ansible automates this entire lifecycle — from provisioning multi-node training clusters with DeepSpeed to deploying vLLM inference servers with A/B testing. As organizations invest in proprietary AI models trained on their domain data, the infrastructure to build and serve these models becomes a strategic asset.