qvest-task/docs/diagrams/ci-cd-workflow.md

CI/CD Workflow with Gitea Actions

This diagram shows the complete CI/CD workflow using Gitea Actions with self-hosted runners, including the automated setup process.

Overview

• Gitea Actions: GitHub Actions-compatible CI/CD built into Gitea
• Self-hosted runners: 2 act_runner instances running as systemd services
• Automated setup: Admin user, runner tokens, and registration fully automated via Ansible
• Test workflow: Integration tests run on every PR to main branch

CI/CD Workflow Diagram

%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#e5e7eb','primaryTextColor':'#111827','primaryBorderColor':'#9ca3af','lineColor':'#111827','secondaryColor':'#d1d5db','tertiaryColor':'#f3f4f6','edgeLabelBackground':'#ffffff','mainBkg':'#f5f5f4','nodeBorder':'#9ca3af','background':'#f5f5f4','clusterBkg':'transparent'},'themeCSS':'.node rect, .node circle, .node ellipse, .node polygon, .node path { filter: none !important; box-shadow: none !important; } .cluster rect { filter: none !important; box-shadow: none !important; } svg { background-color: #f5f5f4 !important; } .cluster-label { background-color: #ffffff !important; padding: 6px 12px !important; border-radius: 4px !important; font-size: 16px !important; font-weight: 700 !important; box-shadow: 0 1px 3px rgba(0,0,0,0.12) !important; border: 1px solid #d1d5db !important; } .edgePath, .edgePath path, .flowchart-link { z-index: 1 !important; }'}}%%

flowchart TB
    Dev([Developer])
    
    subgraph Workflow["CI/CD Workflow"]
        Push[Git Push / PR Created]
        Trigger{Gitea Actions<br/>Workflow Trigger}
        Queue[Job Queued]
        
        subgraph Runners["Self-Hosted Runners"]
            Runner1[act_runner-1<br/>systemd service]
            Runner2[act_runner-2<br/>systemd service]
        end
        
        Pick{Runner<br/>Available?}
        Checkout[📥 Checkout Code]
        Cache[💾 Setup Docker Cache]
        Pull[📥 Pre-pull Test Images<br/>postgres:18.4, nginx:1.27-alpine, alpine:3.19/3.20]
        Test[🧪 Run Integration Tests<br/>scripts/test-integration.sh]
        TestResult{Tests<br/>Pass?}
        Success[✅ Report Success<br/>PR can merge]
        Failure[❌ Report Failure<br/>Upload test logs]
        Artifact[📦 Upload Artifacts<br/>7-day retention]
    end
    
    Dev -->|git push| Push
    Push --> Trigger
    Trigger -->|PR to main| Queue
    Trigger -->|workflow_dispatch| Queue
    Queue --> Pick
    Pick -->|Assigns Job| Runner1
    Pick -->|Assigns Job| Runner2
    Runner1 --> Checkout
    Runner2 --> Checkout
    Checkout --> Cache
    Cache --> Pull
    Pull --> Test
    Test --> TestResult
    TestResult -->|✅ All Pass| Success
    TestResult -->|❌ Any Fail| Failure
    Failure --> Artifact
    
    style Dev fill:#8B5CF6,stroke:#6D28D9,stroke-width:2px,color:#fff
    style Push fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style Trigger fill:#F97316,stroke:#C2410C,stroke-width:2px,color:#111827
    style Queue fill:#F59E0B,stroke:#B45309,stroke-width:2px,color:#111827
    style Pick fill:#F97316,stroke:#C2410C,stroke-width:2px,color:#111827
    style Runner1 fill:#EF4444,stroke:#B91C1C,stroke-width:2px,color:#fff
    style Runner2 fill:#EF4444,stroke:#B91C1C,stroke-width:2px,color:#fff
    style Checkout fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style Cache fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style Pull fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style Test fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style TestResult fill:#F97316,stroke:#C2410C,stroke-width:2px,color:#111827
    style Success fill:#10B981,stroke:#047857,stroke-width:2px,color:#111827
    style Failure fill:#EF4444,stroke:#B91C1C,stroke-width:2px,color:#fff
    style Artifact fill:#6366F1,stroke:#4338CA,stroke-width:2px,color:#fff

Automated Setup Flow

This diagram shows how the runner infrastructure is automatically provisioned and configured.

%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#e5e7eb','primaryTextColor':'#111827','primaryBorderColor':'#9ca3af','lineColor':'#111827','secondaryColor':'#d1d5db','tertiaryColor':'#f3f4f6','edgeLabelBackground':'#ffffff','mainBkg':'#f5f5f4','nodeBorder':'#9ca3af','background':'#f5f5f4','clusterBkg':'transparent'},'themeCSS':'.node rect, .node circle, .node ellipse, .node polygon, .node path { filter: none !important; box-shadow: none !important; } .cluster rect { filter: none !important; box-shadow: none !important; } svg { background-color: #f5f5f4 !important; } .cluster-label { background-color: #ffffff !important; padding: 6px 12px !important; border-radius: 4px !important; font-size: 16px !important; font-weight: 700 !important; box-shadow: 0 1px 3px rgba(0,0,0,0.12) !important; border: 1px solid #d1d5db !important; } .edgePath, .edgePath path, .flowchart-link { z-index: 1 !important; }'}}%%

flowchart TD
    Start([Terraform Apply])
    Secrets[🔐 Create AWS Secrets<br/>DB credentials, Admin credentials]
    EC2[🖥️ Provision EC2 Instance<br/>With IAM role for Secrets Manager]
    
    Ansible([Ansible Playbook])
    Deploy[📦 Deploy Gitea<br/>docker-compose up]
    Wait[⏳ Wait for Gitea<br/>HTTP 200 response]
    CreateUser[👤 Create Admin User<br/>docker exec gitea gitea admin user create]
    DisableChange[🔓 Disable Password Change<br/>UPDATE user SET must_change_password=false]
    GenToken[🎟️ Generate Runner Token<br/>GET /api/v1/admin/runners/registration-token]
    UpdateSecret[💾 Store Token in Secrets Manager<br/>aws secretsmanager update-secret]
    
    DownloadRunner[📥 Download act_runner v0.2.10]
    CreateDirs[📁 Create /etc/act_runner-{1,2}]
    FetchToken[🔍 Fetch Runner Token<br/>from Secrets Manager]
    RegisterRunner[📝 Register Runners<br/>act_runner register --instance http://localhost:3000]
    CreateService[⚙️ Create systemd services<br/>act_runner-1.service, act_runner-2.service]
    StartService[▶️ Enable & Start Services]
    
    Complete([✅ Ready for CI/CD])
    
    Start --> Secrets
    Secrets --> EC2
    EC2 --> Ansible
    
    Ansible --> Deploy
    Deploy --> Wait
    Wait --> CreateUser
    CreateUser --> DisableChange
    DisableChange --> GenToken
    GenToken --> UpdateSecret
    
    UpdateSecret --> DownloadRunner
    DownloadRunner --> CreateDirs
    CreateDirs --> FetchToken
    FetchToken --> RegisterRunner
    RegisterRunner --> CreateService
    CreateService --> StartService
    
    StartService --> Complete
    
    style Start fill:#F59E0B,stroke:#B45309,stroke-width:2px,color:#111827
    style Secrets fill:#8B5CF6,stroke:#6D28D9,stroke-width:2px,color:#fff
    style EC2 fill:#10B981,stroke:#047857,stroke-width:2px,color:#111827
    style Ansible fill:#F59E0B,stroke:#B45309,stroke-width:2px,color:#111827
    style Deploy fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style Wait fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style CreateUser fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style DisableChange fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style GenToken fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style UpdateSecret fill:#8B5CF6,stroke:#6D28D9,stroke-width:2px,color:#fff
    style DownloadRunner fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style CreateDirs fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style FetchToken fill:#8B5CF6,stroke:#6D28D9,stroke-width:2px,color:#fff
    style RegisterRunner fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style CreateService fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style StartService fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
    style Complete fill:#10B981,stroke:#047857,stroke-width:2px,color:#111827

Workflow Configuration

The CI/CD workflow is defined in .gitea/workflows/test.yml:

name: Integration Tests

on:
  pull_request:
    branches: [main]
  workflow_dispatch:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      
      - name: Cache Docker layers
        uses: actions/cache@v4
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: ${{ runner.os }}-buildx-
      
      - name: Pre-pull test images
        run: |
          docker pull postgres:18.4
          docker pull nginx:1.27-alpine
          docker pull alpine:3.19
          docker pull alpine:3.20          
      
      - name: Run integration tests
        run: ./scripts/test-integration.sh
      
      - name: Upload test logs
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: test-logs
          path: /tmp/test-*.log
          retention-days: 7

Test Suite

The scripts/test-integration.sh integration test suite validates:

1. Static validation (2 tests):

   • Script syntax and linting
   • Required executables available

2. Docker-based tests (12 tests):

   • PostgreSQL backup and restore
   • Health check functionality
   • Archive validation (SQL and tar formats)
   • Update simulation workflow
   • Container cleanup and resource management

All tests must pass for a PR to be mergeable.

Key Features

Zero-Configuration CI/CD

• Runners automatically registered during initial deployment
• No manual token management needed
• Runner tokens stored securely in AWS Secrets Manager
• Complete automation from infrastructure provision to working CI/CD

High Availability

• 2 concurrent runners for parallel job execution
• Automatic job distribution by Gitea
• Systemd ensures runners restart on failure

Security

• Runners use local Gitea instance (http://localhost:3000)
• Admin credentials never exposed (CLI-based user creation)
• IAM roles for least-privilege access to AWS resources
• Runner tokens rotated on redeployment

Docker Optimization

• Docker layer caching for faster builds
• Image pre-pulling reduces test execution time
• Shared Docker daemon for all tests

Deployment Commands

# Full deployment (includes runner setup)
make full-deploy

# Update only configuration (re-registers runners if needed)
make configure

# Run tests locally
make test