diff --git a/MIGRATION_STATUS.md b/MIGRATION_STATUS.md new file mode 100644 index 0000000..b6e0148 --- /dev/null +++ b/MIGRATION_STATUS.md @@ -0,0 +1,219 @@ +# Migration Project Status + +**Date:** 2025-12-31 +**Goal:** Migrate services from old prod server to new Ubuntu VM with GitOps deployment + +--- + +## Devices Involved + +| Device | IP | Role | OS | +|--------|-----|------|-----| +| core.localdomain | 192.168.5.34 | Old prod server | Debian/Ubuntu | +| ubuntu-prod | 192.168.5.123 | New prod server | Ubuntu 24.04 | +| NERV-III | Proxmox host | Hypervisor running ubuntu-prod | Proxmox | + +**SSH Access:** +- Old prod: `ssh root@core` +- New prod: `ssh knight@192.168.5.123` (root disabled, use knight + sudo) + +--- + +## What Was Accomplished + +### 1. New Ubuntu VM Provisioned +- Created `ubuntu-prod` VM on NERV-III via Ansible +- Specs: 6 cores, 16GB RAM, 150GB disk +- Configured with Docker, auto security updates (unattended-upgrades) + +### 2. GitOps Repo Created +- **Repo:** https://gitea.ghost.tel/knight/docker-stacks +- **Local path:** `/var/core/docker-stacks` +- Contains 23 service stacks with docker-compose files +- Gitea Actions workflow auto-deploys on push to master + +### 3. Gitea Runner Registered on ubuntu-prod +- `act_runner` v0.2.11 installed at `/usr/local/bin/act_runner` +- Registered with label `ubuntu-prod` +- Running as background process (NOT yet a systemd service) +- Config at: `/home/knight/.runner` +- Log at: `/tmp/act_runner.log` + +### 4. Initial Services Running on ubuntu-prod +``` +traefik - Reverse proxy (v2.10) +dockge - Container management UI (port 5001) +watchtower - Auto container updates +smokeping - Network monitoring +``` + +### 5. GitOps Workflow Tested & Working +- Push to `stacks/**` on master branch triggers deploy +- Workflow copies compose files to `/var/core/{service}/` +- Creates `.env` from `.env.template` using Gitea secrets +- Runs `docker compose up -d` + +--- + +## Repository Structure + +``` +/var/core/docker-stacks/ +├── .gitea/workflows/deploy.yml # GitOps workflow +├── README.md # Setup docs + secrets list +├── scripts/deploy.sh # Manual deploy script +└── stacks/ + ├── authentik/ # SSO provider + ├── bookclub/ # Form mailer + ├── brain/ # Static site + SFTP + ├── changedetection/ # Web change monitor + ├── dockge/ # Container UI + ├── filebrowser/ # Web file manager + ├── ghost/ # Blog + ├── gitea/ # Git server + runner + ├── gollum/ # Wiki + ├── invidious/ # YouTube frontend + ├── memento/ # Custom app + ├── obsidian-tools/ # obbytodo + search + syncthing + ├── perilous/ # Blog + code-server + ├── radicale/ # CalDAV/CardDAV + ├── ramz/ # Go app + ├── registry/ # Docker registry + ├── smokeping/ # Network monitoring + ├── syncthing/ # File sync + ├── traefik/ # Reverse proxy + ├── wallabag/ # Read-later + ├── watchtower/ # Auto updates + ├── xbackbone/ # ShareX server + └── zerotier/ # ZT UI +``` + +**Pruned (not migrating):** immich, planka, chevereto, vikunja + +--- + +## Gitea Secrets Configured + +| Secret | Status | +|--------|--------| +| `DOMAIN` | ✅ Set to `ghost.tel` | +| All others | ❌ Not yet configured | + +See `README.md` in repo for full secrets list needed. + +--- + +## What's Left To Do + +### Immediate +1. **Make runner persistent:** + ```bash + ssh knight@192.168.5.123 + sudo tee /etc/systemd/system/act_runner.service << 'EOF' + [Unit] + Description=Gitea Actions Runner + After=network.target + + [Service] + Type=simple + User=knight + WorkingDirectory=/home/knight + ExecStart=/usr/local/bin/act_runner daemon + Restart=always + + [Install] + WantedBy=multi-user.target + EOF + sudo systemctl enable --now act_runner + ``` + +2. **Add remaining Gitea secrets** (see README.md for full list) + +3. **Add missing config files to repo:** + - `traefik/conf.d/` - middleware configs + - `gollum/config.rb` + - Other service-specific configs + +### Data Migration (per service) +```bash +# Pattern for each service: +# 1. Stop on old prod +ssh root@core "cd /var/core/SERVICE && docker compose down" + +# 2. Rsync data +rsync -avz root@core:/var/core/SERVICE/data/ knight@192.168.5.123:/var/core/SERVICE/data/ + +# 3. Push compose to trigger deploy (or run manually) +# 4. Test +# 5. Update DNS/HAProxy +``` + +### Services by Migration Complexity + +**Easy (stateless or simple volumes):** +- smokeping ✅ (already done) +- watchtower ✅ (already done) +- dockge ✅ (already done) +- traefik ✅ (already done) +- filebrowser +- radicale +- xbackbone +- syncthing + +**Medium (has database):** +- gitea (mariadb) +- ghost (mysql) +- wallabag (mariadb + redis) +- gollum (git-based wiki) +- changedetection + +**Complex (multiple components or custom):** +- authentik (postgres + redis) - DEPLOY EARLY, others depend on it +- invidious (postgres + sig_helper) +- obsidian-tools (syncthing + 2 custom apps) +- memento (needs authentik) +- perilous (build required) +- ramz (build required) +- bookclub (build required) + +--- + +## Ansible Roles Also Created + +During this session, we also created Ansible roles for all services at: +`/var/core/ansible/Ansible/roles/` + +These provide an alternative deployment method if needed, but GitOps is the preferred approach going forward. + +--- + +## Key Commands Reference + +```bash +# Check ubuntu-prod containers +ssh knight@192.168.5.123 "docker ps" + +# Check runner status +ssh knight@192.168.5.123 "ps aux | grep act_runner" + +# Manual deploy a stack +cd /var/core/docker-stacks +./scripts/deploy.sh smokeping + +# Trigger GitOps deploy +cd /var/core/docker-stacks +# edit a file in stacks/ +git add -A && git commit -m "message" && git push + +# View runner log +ssh knight@192.168.5.123 "tail -f /tmp/act_runner.log" +``` + +--- + +## Network Notes + +- `web` docker network exists on ubuntu-prod (external, for traefik) +- Traefik handles SSL via Let's Encrypt (certresolver: http) +- Old prod still running - no DNS changes made yet +- When ready: update HAProxy/DNS to point services to 192.168.5.123