docs: update docs.
All checks were successful
Update Automation Tests / Integration Tests (pull_request) Successful in 33s

This commit is contained in:
aviyadeveloper 2026-06-11 17:40:27 +02:00
parent fbcd561ee8
commit 3821f45d63
6 changed files with 369 additions and 29 deletions

46
ADR.md
View File

@ -117,7 +117,7 @@ This document tracks all significant architectural decisions made during the pro
## ADR-007: SSL Certificates - Let's Encrypt ## ADR-007: SSL Certificates - Let's Encrypt
**Date**: 2026-06-08 **Date**: 2026-06-08 (Updated 2026-06-11)
**Status**: Accepted **Status**: Accepted
**Decision**: Let's Encrypt with certbot **Decision**: Let's Encrypt with certbot
@ -130,6 +130,10 @@ This document tracks all significant architectural decisions made during the pro
**Requirement**: Valid domain name pointing to server **Requirement**: Valid domain name pointing to server
**Domain**: git.poll-streams.com (changed from gitea.poll-streams.com)
**Implementation Note**: Initially encountered Let's Encrypt rate limits (5 certificates per week). Resolved by migrating to a fresh domain identifier (git.poll-streams.com), allowing immediate production certificate issuance. Production certificates obtained successfully.
--- ---
## ADR-008: Update Automation - Diun + Custom Scripts ## ADR-008: Update Automation - Diun + Custom Scripts
@ -167,6 +171,43 @@ This document tracks all significant architectural decisions made during the pro
--- ---
## ADR-012: CI/CD - Gitea Actions with Self-Hosted Runners
**Date**: 2026-06-11
**Status**: Accepted
**Decision**: Use Gitea Actions with self-hosted runners for CI/CD
**Rationale**:
- Native integration with Gitea (no external CI service)
- Self-hosted runners provide full control and security
- GitHub Actions-compatible workflow syntax (familiar, well-documented)
- Enables automated testing before merging changes
- Demonstrates production-grade CI/CD practices
**Implementation**:
- **Runners**: 2x act_runner v0.2.10 instances as systemd services
- **Automation**: Ansible playbook (setup-runner.yml) for reproducible deployment
- **Runner Registration**: Automated via Gitea API with token from AWS Secrets Manager
- **Networking**: Host network mode for job containers to access Gitea
- **Registration URL**: https://git.poll-streams.com (public URL for git clone operations)
- **Workflow**: .gitea/workflows/test.yml runs integration tests on PRs
- **Features**: Docker layer caching, artifact uploads, workflow_dispatch support
**Technical Details**:
- Each runner has dedicated config directory (/etc/act_runner-{1,2})
- Configuration includes host networking to allow job containers to reach services
- Runners registered with public URL to avoid localhost connection issues
- Systemd manages runner lifecycle with automatic restart
**Benefits**:
- Automated quality gates before merging
- Consistent test environment (matches CI exactly)
- Fast feedback on code changes
- Self-contained solution (no external dependencies)
---
## ADR-009: Monitoring - Prometheus + Grafana ## ADR-009: Monitoring - Prometheus + Grafana
**Date**: 2026-06-08 **Date**: 2026-06-08
@ -236,7 +277,8 @@ This document tracks all significant architectural decisions made during the pro
| **Reverse Proxy** | Nginx | Lightweight, standard | | **Reverse Proxy** | Nginx | Lightweight, standard |
| **SSL** | Let's Encrypt | Free, automated, professional | | **SSL** | Let's Encrypt | Free, automated, professional |
| **DNS** | Route 53 | AWS-native | | **DNS** | Route 53 | AWS-native |
| **Updates** | Watchtower | Docker-native automation | | **Updates** | Diun + Scripts | Per-container policies, backup/rollback |
| **CI/CD** | Gitea Actions | Self-hosted runners, native integration |
| **Backups** | Scripts + S3 | Custom, controlled | | **Backups** | Scripts + S3 | Custom, controlled |
| **Monitoring** | Prometheus + Grafana | Industry standard | | **Monitoring** | Prometheus + Grafana | Industry standard |
| **Logging** | Loki + Promtail | Lightweight, integrated | | **Logging** | Loki + Promtail | Lightweight, integrated |

View File

@ -104,7 +104,8 @@ This phase implements the automated, reproducible Gitea installation.
### 3.3 Reverse Proxy Configuration ✅ ### 3.3 Reverse Proxy Configuration ✅
- ✅ Nginx 1.27-alpine deployed via Docker Compose - ✅ Nginx 1.27-alpine deployed via Docker Compose
- ✅ Let's Encrypt SSL certificate obtained via certbot - ✅ Let's Encrypt SSL certificate obtained via certbot (production)
- ✅ Domain: git.poll-streams.com (migrated to avoid rate limits)
- ✅ Two-stage nginx config (HTTP-only for ACME, then HTTPS) - ✅ Two-stage nginx config (HTTP-only for ACME, then HTTPS)
- ✅ SSL termination at nginx, proxy to Gitea on port 3000 - ✅ SSL termination at nginx, proxy to Gitea on port 3000
- ✅ HTTP to HTTPS redirect configured - ✅ HTTP to HTTPS redirect configured
@ -114,7 +115,7 @@ This phase implements the automated, reproducible Gitea installation.
### 3.4 Testing ✅ ### 3.4 Testing ✅
- ✅ HTTPS access verified: https://git.poll-streams.com - ✅ HTTPS access verified: https://git.poll-streams.com
- ✅ Valid SSL certificate (Let's Encrypt) - ✅ Valid SSL certificate (Let's Encrypt production)
- ✅ HTTP → HTTPS redirect working - ✅ HTTP → HTTPS redirect working
- ✅ Gitea web interface accessible and functional - ✅ Gitea web interface accessible and functional
- ✅ User account created, repository created - ✅ User account created, repository created
@ -191,11 +192,24 @@ This phase implements automated update mechanisms for Gitea and related componen
- ✅ Diun monitoring confirmed (4 containers) - ✅ Diun monitoring confirmed (4 containers)
- ✅ Update workflow diagram created - ✅ Update workflow diagram created
### 4.7 CI/CD Implementation ✅
- ✅ Gitea Actions enabled on instance
- ✅ Self-hosted runners deployed (2x act_runner v0.2.10)
- ✅ Runner automation via Ansible (setup-runner.yml)
- ✅ Systemd services for runner management
- ✅ Host networking configuration for job containers
- ✅ CI workflow created (.gitea/workflows/test.yml)
- ✅ Automated testing on pull requests
- ✅ Docker layer caching for performance
- ✅ Artifact upload on test failure
- ✅ Full CI/CD pipeline tested and operational
### Goals: ### Goals:
- ✅ Automated update system operational - ✅ Automated update system operational
- ✅ Update process tested and validated on live system - ✅ Update process tested and validated on live system
- ✅ Rollback procedure implemented and tested - ✅ Rollback procedure implemented and tested
- ✅ Quality gate for CI/local environments - ✅ Quality gate for CI/local environments
- ✅ CI/CD pipeline with self-hosted runners
- ✅ Documentation complete (workflow diagram) - ✅ Documentation complete (workflow diagram)
**Implementation Summary:** **Implementation Summary:**
@ -205,9 +219,10 @@ This phase implements automated update mechanisms for Gitea and related componen
- Pre-update backups with automatic rollback on failure - Pre-update backups with automatic rollback on failure
- Certificate renewal automation - Certificate renewal automation
- Comprehensive testing framework - Comprehensive testing framework
- Visual workflow documentation - CI/CD with Gitea Actions and 2 self-hosted runners
- Visual workflow documentation (including CI/CD flow)
**Phase 4 Complete!** Update automation fully operational with safety mechanisms. **Phase 4 Complete!** Update automation and CI/CD fully operational with safety mechanisms.
--- ---

View File

@ -12,46 +12,72 @@ graph TB
subgraph EC2["EC2 Instance"] subgraph EC2["EC2 Instance"]
subgraph Docker["Docker Compose"] subgraph Docker["Docker Compose"]
Nginx[Nginx<br/>Port 80, 443] Nginx[Nginx<br/>Port 80, 443]
Gitea[Gitea<br/>Port 3000] Gitea[Gitea<br/>Port 3000, 2222]
Postgres[(PostgreSQL<br/>Port 5432)] Postgres[(PostgreSQL<br/>Port 5432)]
Watchtower[Watchtower<br/>Auto-updater] Certbot[Certbot<br/>SSL Renewal]
DIUN[DIUN<br/>Update Monitor]
Nginx -->|Reverse Proxy| Gitea Nginx -->|Reverse Proxy| Gitea
Gitea -->|Database Connection| Postgres Gitea -->|Database Connection| Postgres
Watchtower -.->|Monitors & Updates| Nginx DIUN -.->|Monitors for Updates| Nginx
Watchtower -.->|Monitors & Updates| Gitea DIUN -.->|Monitors for Updates| Gitea
DIUN -.->|Monitors for Updates| Postgres
Certbot -.->|Renews Certificates| Nginx
end
subgraph Systemd["Systemd Services"]
Runner1[act_runner-1<br/>CI/CD Runner]
Runner2[act_runner-2<br/>CI/CD Runner]
Runner1 -.->|Executes Workflows| Gitea
Runner2 -.->|Executes Workflows| Gitea
end end
end end
User -->|HTTPS| Nginx User -->|HTTPS| Nginx
LetsEncrypt -.->|Certbot Renewal| Nginx User -->|Git SSH| Gitea
LetsEncrypt -.->|Certificate Authority| Certbot
style EC2 fill:#e5e7eb,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5 style EC2 fill:#e5e7eb,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
style Docker fill:#d1d5db,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5 style Docker fill:#d1d5db,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
style Systemd fill:#d1d5db,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
style Nginx fill:#10B981,stroke:#333,stroke-width:1px,color:#fff style Nginx fill:#10B981,stroke:#333,stroke-width:1px,color:#fff
style Gitea fill:#3B82F6,stroke:#333,stroke-width:1px,color:#fff style Gitea fill:#3B82F6,stroke:#333,stroke-width:1px,color:#fff
style Postgres fill:#8B5CF6,stroke:#333,stroke-width:1px,color:#fff style Postgres fill:#8B5CF6,stroke:#333,stroke-width:1px,color:#fff
style Watchtower fill:#F59E0B,stroke:#333,stroke-width:1px,color:#fff style DIUN fill:#F59E0B,stroke:#333,stroke-width:1px,color:#fff
style Certbot fill:#6366F1,stroke:#333,stroke-width:1px,color:#fff
style Runner1 fill:#EF4444,stroke:#333,stroke-width:1px,color:#fff
style Runner2 fill:#EF4444,stroke:#333,stroke-width:1px,color:#fff
``` ```
## Components ## Components
### Docker Containers
- **Nginx**: Reverse proxy handling SSL termination and routing to Gitea - **Nginx**: Reverse proxy handling SSL termination and routing to Gitea
- **Gitea**: Git server application (main service) - **Gitea**: Git server application with Actions enabled (HTTP: 3000, SSH: 2222)
- **PostgreSQL**: Database storing repositories metadata, users, issues - **PostgreSQL**: Database storing repositories metadata, users, issues
- **Watchtower**: Monitors Docker Hub for image updates, automatically pulls and restarts containers - **DIUN**: Monitors Docker Hub for image updates, sends email notifications
- **Certbot**: Handles Let's Encrypt SSL certificate renewal
### Systemd Services
- **act_runner-1**: First Gitea Actions runner for CI/CD workflows
- **act_runner-2**: Second Gitea Actions runner for CI/CD workflows
## Container Communication ## Container Communication
- All containers in the same Docker network - All containers in the same Docker network (`gitea-network`)
- Nginx proxies HTTPS requests to Gitea's internal port 3000 - Nginx proxies HTTPS requests to Gitea's internal port 3000
- Gitea connects to PostgreSQL via container name - Gitea connects to PostgreSQL via container name (`postgres`)
- Watchtower runs on schedule, checking for updates - DIUN monitors containers based on labels (`diun.enable=true`)
- Let's Encrypt certbot renews certificates automatically (via nginx container or separate container) - Certbot shares volumes with nginx for certificate storage
- Runners connect to Gitea via `http://localhost:3000`
## Data Persistence ## Data Persistence
Docker volumes ensure data survives container restarts: Docker volumes ensure data survives container restarts:
- `gitea_data`: Git repositories and uploads - `gitea-data`: Git repositories and uploads
- `postgres_data`: Database files - `gitea_postgres-data`: PostgreSQL database files
- `certbot-etc`: Let's Encrypt certificates
- `certbot-var`: Certbot working directory
- `web-root`: ACME challenge files for SSL verification

View File

@ -8,12 +8,17 @@ This diagram shows the high-level AWS resources and their relationships.
graph TB graph TB
Internet([Internet/Users]) Internet([Internet/Users])
Route53[Route 53<br/>DNS] Route53[Route 53<br/>DNS]
EC2[EC2 Instance<br/>Docker Host] EC2[EC2 Instance<br/>Docker Host + Runners]
S3[(S3 Bucket<br/>Backups)] S3[(S3 Bucket<br/>Backups)]
Secrets[AWS Secrets Manager<br/>DB/Admin Credentials]
IAM[IAM Role<br/>EC2 Permissions]
Internet -->|HTTPS| Route53 Internet -->|HTTPS| Route53
Route53 -->|DNS Resolution| EC2 Route53 -->|DNS Resolution| EC2
EC2 -->|Backup Upload| S3 EC2 -->|Backup Upload| S3
EC2 -->|Fetch Credentials| Secrets
IAM -.->|Attached to| EC2
EC2 -->|Update Runner Token| Secrets
subgraph AWS["AWS Account"] subgraph AWS["AWS Account"]
subgraph VPC["VPC"] subgraph VPC["VPC"]
@ -21,6 +26,8 @@ graph TB
end end
Route53 Route53
S3 S3
Secrets
IAM
end end
style AWS fill:#e5e7eb,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5 style AWS fill:#e5e7eb,stroke:#4b5563,stroke-width:2px,stroke-dasharray: 5 5
@ -29,18 +36,24 @@ graph TB
style EC2 fill:#10B981,stroke:#333,stroke-width:1px,color:#fff style EC2 fill:#10B981,stroke:#333,stroke-width:1px,color:#fff
style S3 fill:#F97316,stroke:#333,stroke-width:1px,color:#fff style S3 fill:#F97316,stroke:#333,stroke-width:1px,color:#fff
style Route53 fill:#6366F1,stroke:#333,stroke-width:1px,color:#fff style Route53 fill:#6366F1,stroke:#333,stroke-width:1px,color:#fff
style Secrets fill:#8B5CF6,stroke:#333,stroke-width:1px,color:#fff
style IAM fill:#F59E0B,stroke:#333,stroke-width:1px,color:#fff
``` ```
## Components ## Components
- **Route 53**: DNS service that points domain to EC2 instance - **Route 53**: DNS service that points domain to EC2 instance
- **EC2 Instance**: Single VM running Docker with all application containers - **EC2 Instance**: Single VM running Docker containers + 2 Gitea Actions runners (systemd services)
- **S3 Bucket**: Storage for database and application backups - **S3 Bucket**: Storage for database and application backups (with versioning)
- **AWS Secrets Manager**: Stores DB credentials, admin credentials, SES SMTP credentials, runner tokens
- **IAM Role**: EC2 instance profile with permissions for S3, Secrets Manager read/update
- **VPC**: Isolated network containing EC2 instance - **VPC**: Isolated network containing EC2 instance
## Traffic Flow ## Traffic Flow
1. User accesses `gitea.yourdomain.com` 1. User accesses `git.poll-streams.com`
2. Route 53 resolves to EC2 public IP 2. Route 53 resolves to EC2 public IP
3. Request hits EC2 (nginx handles SSL, proxies to Gitea) 3. Request hits EC2 (nginx handles SSL, proxies to Gitea)
4. EC2 regularly backs up data to S3 4. EC2 regularly backs up data to S3
5. Ansible fetches credentials from Secrets Manager during deployment
6. Gitea generates runner token via API, stored back in Secrets Manager

View File

@ -0,0 +1,242 @@
# CI/CD Workflow with Gitea Actions
This diagram shows the complete CI/CD workflow using Gitea Actions with self-hosted runners, including the automated setup process.
## Overview
- **Gitea Actions**: GitHub Actions-compatible CI/CD built into Gitea
- **Self-hosted runners**: 2 act_runner instances running as systemd services
- **Automated setup**: Admin user, runner tokens, and registration fully automated via Ansible
- **Test workflow**: Integration tests run on every PR to main branch
## CI/CD Workflow Diagram
```mermaid
%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#e5e7eb','primaryTextColor':'#111827','primaryBorderColor':'#9ca3af','lineColor':'#111827','secondaryColor':'#d1d5db','tertiaryColor':'#f3f4f6','edgeLabelBackground':'#ffffff','mainBkg':'#f5f5f4','nodeBorder':'#9ca3af','background':'#f5f5f4','clusterBkg':'transparent'},'themeCSS':'.node rect, .node circle, .node ellipse, .node polygon, .node path { filter: none !important; box-shadow: none !important; } .cluster rect { filter: none !important; box-shadow: none !important; } svg { background-color: #f5f5f4 !important; } .cluster-label { background-color: #ffffff !important; padding: 6px 12px !important; border-radius: 4px !important; font-size: 16px !important; font-weight: 700 !important; box-shadow: 0 1px 3px rgba(0,0,0,0.12) !important; border: 1px solid #d1d5db !important; } .edgePath, .edgePath path, .flowchart-link { z-index: 1 !important; }'}}%%
flowchart TB
Dev([Developer])
subgraph Workflow["CI/CD Workflow"]
Push[Git Push / PR Created]
Trigger{Gitea Actions<br/>Workflow Trigger}
Queue[Job Queued]
subgraph Runners["Self-Hosted Runners"]
Runner1[act_runner-1<br/>systemd service]
Runner2[act_runner-2<br/>systemd service]
end
Pick{Runner<br/>Available?}
Checkout[📥 Checkout Code]
Cache[💾 Setup Docker Cache]
Pull[📥 Pre-pull Test Images<br/>postgres:18.4, nginx:1.27-alpine, alpine:3.19/3.20]
Test[🧪 Run Integration Tests<br/>scripts/test-update.sh]
TestResult{Tests<br/>Pass?}
Success[✅ Report Success<br/>PR can merge]
Failure[❌ Report Failure<br/>Upload test logs]
Artifact[📦 Upload Artifacts<br/>7-day retention]
end
Dev -->|git push| Push
Push --> Trigger
Trigger -->|PR to main| Queue
Trigger -->|workflow_dispatch| Queue
Queue --> Pick
Pick -->|Assigns Job| Runner1
Pick -->|Assigns Job| Runner2
Runner1 --> Checkout
Runner2 --> Checkout
Checkout --> Cache
Cache --> Pull
Pull --> Test
Test --> TestResult
TestResult -->|✅ All Pass| Success
TestResult -->|❌ Any Fail| Failure
Failure --> Artifact
style Dev fill:#8B5CF6,stroke:#6D28D9,stroke-width:2px,color:#fff
style Push fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style Trigger fill:#F97316,stroke:#C2410C,stroke-width:2px,color:#111827
style Queue fill:#F59E0B,stroke:#B45309,stroke-width:2px,color:#111827
style Pick fill:#F97316,stroke:#C2410C,stroke-width:2px,color:#111827
style Runner1 fill:#EF4444,stroke:#B91C1C,stroke-width:2px,color:#fff
style Runner2 fill:#EF4444,stroke:#B91C1C,stroke-width:2px,color:#fff
style Checkout fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style Cache fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style Pull fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style Test fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style TestResult fill:#F97316,stroke:#C2410C,stroke-width:2px,color:#111827
style Success fill:#10B981,stroke:#047857,stroke-width:2px,color:#111827
style Failure fill:#EF4444,stroke:#B91C1C,stroke-width:2px,color:#fff
style Artifact fill:#6366F1,stroke:#4338CA,stroke-width:2px,color:#fff
```
## Automated Setup Flow
This diagram shows how the runner infrastructure is automatically provisioned and configured.
```mermaid
%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#e5e7eb','primaryTextColor':'#111827','primaryBorderColor':'#9ca3af','lineColor':'#111827','secondaryColor':'#d1d5db','tertiaryColor':'#f3f4f6','edgeLabelBackground':'#ffffff','mainBkg':'#f5f5f4','nodeBorder':'#9ca3af','background':'#f5f5f4','clusterBkg':'transparent'},'themeCSS':'.node rect, .node circle, .node ellipse, .node polygon, .node path { filter: none !important; box-shadow: none !important; } .cluster rect { filter: none !important; box-shadow: none !important; } svg { background-color: #f5f5f4 !important; } .cluster-label { background-color: #ffffff !important; padding: 6px 12px !important; border-radius: 4px !important; font-size: 16px !important; font-weight: 700 !important; box-shadow: 0 1px 3px rgba(0,0,0,0.12) !important; border: 1px solid #d1d5db !important; } .edgePath, .edgePath path, .flowchart-link { z-index: 1 !important; }'}}%%
flowchart TD
Start([Terraform Apply])
Secrets[🔐 Create AWS Secrets<br/>DB credentials, Admin credentials]
EC2[🖥️ Provision EC2 Instance<br/>With IAM role for Secrets Manager]
Ansible([Ansible Playbook])
Deploy[📦 Deploy Gitea<br/>docker-compose up]
Wait[⏳ Wait for Gitea<br/>HTTP 200 response]
CreateUser[👤 Create Admin User<br/>docker exec gitea gitea admin user create]
DisableChange[🔓 Disable Password Change<br/>UPDATE user SET must_change_password=false]
GenToken[🎟️ Generate Runner Token<br/>GET /api/v1/admin/runners/registration-token]
UpdateSecret[💾 Store Token in Secrets Manager<br/>aws secretsmanager update-secret]
DownloadRunner[📥 Download act_runner v0.2.10]
CreateDirs[📁 Create /etc/act_runner-{1,2}]
FetchToken[🔍 Fetch Runner Token<br/>from Secrets Manager]
RegisterRunner[📝 Register Runners<br/>act_runner register --instance http://localhost:3000]
CreateService[⚙️ Create systemd services<br/>act_runner-1.service, act_runner-2.service]
StartService[▶️ Enable & Start Services]
Complete([✅ Ready for CI/CD])
Start --> Secrets
Secrets --> EC2
EC2 --> Ansible
Ansible --> Deploy
Deploy --> Wait
Wait --> CreateUser
CreateUser --> DisableChange
DisableChange --> GenToken
GenToken --> UpdateSecret
UpdateSecret --> DownloadRunner
DownloadRunner --> CreateDirs
CreateDirs --> FetchToken
FetchToken --> RegisterRunner
RegisterRunner --> CreateService
CreateService --> StartService
StartService --> Complete
style Start fill:#F59E0B,stroke:#B45309,stroke-width:2px,color:#111827
style Secrets fill:#8B5CF6,stroke:#6D28D9,stroke-width:2px,color:#fff
style EC2 fill:#10B981,stroke:#047857,stroke-width:2px,color:#111827
style Ansible fill:#F59E0B,stroke:#B45309,stroke-width:2px,color:#111827
style Deploy fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style Wait fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style CreateUser fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style DisableChange fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style GenToken fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style UpdateSecret fill:#8B5CF6,stroke:#6D28D9,stroke-width:2px,color:#fff
style DownloadRunner fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style CreateDirs fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style FetchToken fill:#8B5CF6,stroke:#6D28D9,stroke-width:2px,color:#fff
style RegisterRunner fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style CreateService fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style StartService fill:#3B82F6,stroke:#1D4ED8,stroke-width:2px,color:#fff
style Complete fill:#10B981,stroke:#047857,stroke-width:2px,color:#111827
```
## Workflow Configuration
The CI/CD workflow is defined in `.gitea/workflows/test.yml`:
```yaml
name: Integration Tests
on:
pull_request:
branches: [main]
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Cache Docker layers
uses: actions/cache@v4
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: ${{ runner.os }}-buildx-
- name: Pre-pull test images
run: |
docker pull postgres:18.4
docker pull nginx:1.27-alpine
docker pull alpine:3.19
docker pull alpine:3.20
- name: Run integration tests
run: ./scripts/test-update.sh
- name: Upload test logs
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-logs
path: /tmp/test-*.log
retention-days: 7
```
## Test Suite
The `scripts/test-update.sh` integration test suite validates:
1. **Static validation** (2 tests):
- Script syntax and linting
- Required executables available
2. **Docker-based tests** (12 tests):
- PostgreSQL backup and restore
- Health check functionality
- Archive validation (SQL and tar formats)
- Update simulation workflow
- Container cleanup and resource management
All tests must pass for a PR to be mergeable.
## Key Features
### Zero-Configuration CI/CD
- Runners automatically registered during initial deployment
- No manual token management needed
- Runner tokens stored securely in AWS Secrets Manager
- Complete automation from infrastructure provision to working CI/CD
### High Availability
- 2 concurrent runners for parallel job execution
- Automatic job distribution by Gitea
- Systemd ensures runners restart on failure
### Security
- Runners use local Gitea instance (`http://localhost:3000`)
- Admin credentials never exposed (CLI-based user creation)
- IAM roles for least-privilege access to AWS resources
- Runner tokens rotated on redeployment
### Docker Optimization
- Docker layer caching for faster builds
- Image pre-pulling reduces test execution time
- Shared Docker daemon for all tests
## Deployment Commands
```bash
# Full deployment (includes runner setup)
make full-deploy
# Update only configuration (re-registers runners if needed)
make configure
# Run tests locally
make test
```

View File

@ -58,17 +58,19 @@ graph TB
**EC2 Security Group**: **EC2 Security Group**:
- **Inbound Rules**: - **Inbound Rules**:
- Port 22 (SSH): From admin IP only (for management) - Port 22 (SSH): From admin IP only (for management)
- Port 80 (HTTP): From 0.0.0.0/0 (redirects to HTTPS) - Port 80 (HTTP): From 0.0.0.0/0 (redirects to HTTPS, ACME challenge)
- Port 443 (HTTPS): From 0.0.0.0/0 (Gitea access) - Port 443 (HTTPS): From 0.0.0.0/0 (Gitea web access)
- Port 2222 (Git SSH): From 0.0.0.0/0 (Git push/pull via SSH)
- **Outbound Rules**: - **Outbound Rules**:
- All traffic: To 0.0.0.0/0 (for updates, backups to S3) - All traffic: To 0.0.0.0/0 (for updates, backups to S3, Secrets Manager)
## Security Considerations ## Security Considerations
1. **SSH Access**: Restricted to specific admin IP address (your IP) 1. **SSH Access**: Restricted to specific admin IP address (your IP)
2. **HTTP/HTTPS**: Open to internet (required for Gitea web access) 2. **HTTP/HTTPS**: Open to internet (required for Gitea web access)
3. **No Direct Gitea Access**: Port 3000 not exposed; only nginx on 80/443 3. **Git SSH**: Port 2222 exposed for Git operations over SSH
4. **Outbound**: Allowed for Docker image pulls, package updates, S3 backups 4. **No Direct Gitea HTTP Access**: Port 3000 not exposed; only nginx on 80/443
5. **Outbound**: Allowed for Docker image pulls, package updates, S3 backups, AWS API calls
## Traffic Flow ## Traffic Flow