vacuum-wall/AGENTS.md

# Vacuum Wall — Agent Instructions

## What This Is

SSL proxy / firewall appliance. Python 3.13+ Flask SPA behind nginx reverse proxy.
Deploys on Debian 13 (trixie). Serves from repo root by default.

## Architecture

```
Client ──→ nginx (TLS; basic auth on basic-authed proxy domains only) ──→ Flask (127.0.0.1:9090)
Flask  ──→ daemon/client.py (Unix socket) ──→ vacuum-walld (aiohttp, daemon.sock)
vacuum-walld ──→ daemon/handlers/*.py ──→ sudo <cmd> ──→ system service
```

Blueprints are thin proxies — they never call `lib/` directly. All operations flow
through the daemon client over a Unix socket.

**Authentication:** the management UI gets **no** nginx-level `auth_basic` — the mgmt
server block's `location /` is a bare proxy and `/ws` is `auth_basic off`. Management
auth is the Flask-layer JWT middleware (`POST /api/auth/login` →
`Authorization: Bearer <token>`; public paths: static files, `/vendor/`, auth endpoints)
plus the daemon WS handshake (raw JWT as the `Sec-WebSocket-Protocol` subprotocol).
Basic auth (`.htpasswd`) renders **only** for proxy domains whose
`config/nginx/config.json` has an `auth` block — never for the management domain
(see `docs/security.md`, "Management Interface").

### Two-User Model with Shared Group

- **`vacuum-walld`** (daemon user): runs the privileged background daemon with `NOPASSWD sudo` whitelist (`/etc/sudoers.d/vacuum-walld`). Owns socket. Primary group is the WebUI user's primary group. Daemon user name is derived: `USER_NAME` + `d`.
- **WebUI user** (default: repo owner in `--dev` mode): runs the Flask process with **zero sudo** access. Communicates with the daemon via Unix socket.
- **Shared group**: both users share the WebUI user's primary group. Socket is `vacuum-walld:<group>` with mode `0660`.

### Code Layout

- `webui/server.py` — Flask app entry point. **Only** file that creates the `app`. SPA root route (`/`) serves `index.html` (no templating). All other paths return 404.
- `webui/api/*.py` — Flask blueprints, one per subsystem. Routes prefix `/api/<subsystem>/`. All call `daemon.client` instead of `lib/` directly.
- `webui/api/common.py` — Shared `_ok()` / `_error()` response helpers used by all blueprints.
- `daemon/server.py` — aiohttp server, route registry, batch routing, WebSocket broadcast, state refresh, periodic polling.
- `daemon/client.py` — Sync HTTP client over Unix socket using `requests_unixsocket.Session`.
- `daemon/iface.py` — **Single source of truth** for all daemon API endpoints. Every endpoint is a frozen `(method, path)` tuple. Renaming here auto-updates both server registry and client calls.
- `daemon/handlers/*.py` — Privileged operation handlers. All mutating `sudo` calls live here.
- `daemon/collectors/` — Per-subsystem state collectors (7 modules: firewall, dnsmasq, nginx, acme, wireguard, networkd, system). Read-only `sudo` queries that populate `lib.state`. Imported for their registration side-effect; `daemon/server.py` imports the package before the first `populate()`.
- `lib/state.py` — In-memory state store with per-subsystem collectors. Populated at daemon startup, refreshed on mutation/poll. Backs the WebSocket push stream: `get_snapshot()` (full state on WS connect), `poll()` two-layer diff (structural `versions` broadcast vs volatile-only `tick` broadcast, per-subsystem, each carrying the full subsystem data), `register_volatile(subsystem, keys)` to mark volatile fields, `get_versions()`/`bump()`. Per-subsystem poll intervals via `_DEFAULT_POLL_INTERVALS` (system 1s, firewall 30s, wireguard/dnsmasq/networkd 10s, nginx 60s, acme 300s).
- `lib/common.py` — Shared utilities: `run()`, `run_proc()`, `load_json()`, `save_json()`, `deep_merge()`, `ensure_dirs()`, `config_hash()`, `validate_interface_name()`, plus the apply-bookkeeping helpers `stamp_applied()` / `strip_apply_meta()` / `compute_pending()` / `deep_diff()` / `revert_to_applied()` (the `_last_applied_hash` / `_last_applied_config` keys every config-backed subsystem uses for pending-change detection and cancel-all).
- `lib/logging.py` — Logging setup used by both webui and daemon. Reads `VACUUM_WALL_LOG_LEVEL`.
- `lib/sync.py` — Cross-subsystem sync event bus (in-process pub/sub); handlers emit events on mutation and subscribers refresh affected subsystems.
- `lib/system_import.py` — Startup system-config import/reconcile: parses native config sources and merges them into the declarative JSON on daemon start.
- `lib/bootstrap.py` — Daemon-startup filesystem bootstrap, run **after** `system_import.import_all()` (which must see absent config files to adopt live state on first start) and before the first state collection: creates the runtime `config/`+`data/` directories and persists the one-shot nginx legacy-format migration. Never creates config files (reads stay pure; files appear on first `save_config`).
- `lib/schema.py` — TypedDict state schemas for the per-subsystem state payloads.
- `lib/*.py` — Backend modules (parsing, config, shared logic). Full type hints and `__all__` exports. No sudo calls.
- `vendor/` — Vendored scripts and JS libraries (`acme.sh`, `htm`).
- `data/` — Runtime artifacts (generated .confs, `.htpasswd`, ACME certs, firewall backup, dnsmasq fragments).
- `config/<subsystem>/config.json` — Declarative JSON configs (source of truth).
- `system/` — System file templates. `systemd/` (units installed to `/etc/systemd/system/`), `sudoers.d/`, `nginx/`.

Project uses `.venv`. Install deps with `pip install -e .` (from `pyproject.toml`).
`__init__.py` files in `webui/`, `lib/`, and `daemon/` are intentionally empty.

### Frontend (hoover)

Custom reactive SPA framework at `webui/static/hoover/`. See `docs/hoover.md` for full API reference.

Conventions:
- All imports from `/static/hoover/index.js` (barrel export).
- Pages in `webui/static/pages/` export `definePage({ init, subscribe, load, render })`.
- Bootstrap: `webui/static/app.js` mounts two render roots (`#sidebar`, `#main`), then `connect()` for WS.
- `h()` builds VNodes with `on:click` prefix. `html` tag (htm) templates use camelCase `onClick` (adapter translates).
- State always has `loading`, `refreshing`, `error` plus data. `load()` receives `(state, abortController, entry)`.
- `openModal` + `formModal` for dialogs; `apiSubmit()` for form submission.
- No build step — ES modules served raw. Cache controlled via HTTP headers.

### Daemon Endpoints

- Unix socket at `data/daemon.sock` (configurable via `VACUUM_WALLD_SOCKET`)
- WebSocket at `127.0.0.1:9091` (configurable via `VACUUM_WALLD_WS_PORT`) for real-time state streaming: full `snapshot` on connect, then per-subsystem data-carrying `versions` (structural) / `tick` (volatile-only) deltas. The client patches reactive models in place via `modelSet()` — no HTTP round-trip for auto-refresh.
- Periodic polling per subsystem via `lib.state._DEFAULT_POLL_INTERVALS`, overridable with `VACUUM_WALL_POLL_INTERVALS` env var (format `subsystem:seconds,subsystem:seconds`)
- Start as `python -m daemon.server` or via the `vacuum-walld` console script

## Environment Variables

- `VACUUM_WALL_DEV` — dev mode flag; disables aggressive static asset caching
- `VACUUM_WALL_LOG_LEVEL` — log level (default `INFO`)
- `VACUUM_WALLD_SOCKET` — override daemon socket path (default `data/daemon.sock`)
- `VACUUM_WALLD_WS_PORT` — override WebSocket port (default `9091`)
- `VACUUM_WALL_POLL_INTERVALS` — override poll intervals, e.g. `firewall:60,wireguard:5`
- `VACUUM_WALL_EXTERNAL_IP_URL` — custom URL for external IP detection (acme handler)
- `VACUUM_WALL_SEED_BUILTIN_ADMIN` — set to `0` to skip the last-resort builtin admin seed in `get_db()`. The seed only runs on a completely empty DB (no users); `scripts/bootstrap_auth.py` always sets this since bootstrap creates the operator user itself.

## Local Dev

```bash
# Setup
python3 -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"
bash scripts/update-vendor.sh        # fetches acme.sh + htm.js

# Start (Flask only, binds 127.0.0.1:9090)
.venv/bin/python webui/server.py
```

Reload running Flask via SIGHUP (auto-reloads `webui.*` and `lib.*` modules, then SIGTERM restart).

## Blueprint / Handler / lib Mapping

| Blueprint             | URL Prefix          | Handler                    | lib Module      |
|-----------------------|---------------------|----------------------------|-----------------|
| `webui/api/firewall`  | `/api/firewall/`    | `daemon/handlers/firewall` | `lib.firewall`  |
| `webui/api/dhcp`      | `/api/dhcp/`        | `daemon/handlers/dnsmasq`  | `lib.dnsmasq`   |
| `webui/api/proxy`     | `/api/proxy/`       | `daemon/handlers/nginx`    | `lib.nginx`     |
| `webui/api/certs`     | `/api/certs/`       | `daemon/handlers/acme`     | `lib.acme`      |
| `webui/api/wireguard` | `/api/wireguard/`   | `daemon/handlers/wireguard`| `lib.wireguard` |
| `webui/api/network`   | `/api/network/`     | `daemon/handlers/network`  | `lib.network`   |
| `webui/api/logs`      | `/api/logs/`        | `daemon/handlers/logs`     | —               |
| `webui/api/status`    | `/api/status/`      | `daemon/handlers/status`   | —               |

## Privileged Operations

`daemon/handlers/*.py` call `sudo` for everything that touches system services. Whitelist is `system/sudoers.d/vacuum-walld`.

**acme.sh must never run as root** — always as the service user via `sudo -u`.

Pattern for mutations: write JSON → render native config → `sudo <cmd>` to apply.
Adding a new privileged command requires a sudoers entry **and** the `daemon/handlers/` code.

**Config reads are pure.** Every `lib/<subsystem>.get_config()` is a side-effect-free
read (returns in-memory defaults when the file is missing; nginx applies its
legacy-format migration in memory). State collectors therefore never write to
disk — filesystem setup (runtime dirs, one-shot nginx migration) happens once at
daemon startup in `lib/bootstrap.py` (after `system_import.import_all()`).

**Firewall interface-coverage invariant.** Every network-managed interface
(`lo`/`wg*` excluded) must be covered by a zone in `config/firewall/config.json`
or listed under the top-level `unmanaged` key. The config is the source of truth
for zone interfaces (an omitted `interfaces` key = empty list; no hands-off
zones). Enforced at save time (`POST`/`PATCH /firewall/config` → 400) and apply
time (`POST /firewall/config/apply` → 409, `force: true` overrides) via the pure
`lib.firewall.validate_coverage()`. Live drift is advisory only
(`uncovered_interfaces` state field). See `docs/config.md`.

## API Response Contract

- Success: `{"ok": true, "data": <value>}` — `_ok(data)` (Flask) or `ok(data)` (aiohttp)
- Error: `{"ok": false, "error": "msg"}` — `_error(msg, code)` (Flask) or `error(msg, code)` (aiohttp)
- `acme.issue()` / `acme.renew()` raise `RuntimeError` on failure — API layer wraps in try/except
- HTTP codes: `400` bad request, `404` not found, `409` conflict, `500` internal failure

## Deploy

`scripts/install.sh` is the single deploy script. Run as root. CLI flags take precedence over env vars.
`MGMT_PASS` is strictly required — it is the **SQLite DB** password for the initial `admin`
user (full `rw` on all subsystems; default username `admin`), **not** an nginx htpasswd.
`MGMT_DOMAIN` auto-detected from hostname.

### Service Start Order

`firewalld` → `avahi-daemon` → `dnsmasq` → `vacuum-walld` → `vacuum-wall`

## Lint and Tests

**Linter / formatter:** Ruff (`ruff check` + `ruff format`). Config in `pyproject.toml`.

**Tests:** pytest in `tests/` (27 test files). All subprocess calls are mocked — no system services required.

```bash
.venv/bin/ruff check lib/ webui/ tests/   # lint
.venv/bin/ruff format lib/ webui/ tests/   # format
.venv/bin/python -m pytest tests/ -v       # test
```

## Docs

`docs/` contains the authoritative reference for each subsystem. Read the relevant doc before reasoning about a subsystem.

| Doc | Contents |
|-----|----------|
| `docs/architecture.md` | Request flow, subsystem communication, two-user model, zone model, state management |
| `docs/security.md` | Privilege model, sudo whitelist, systemd hardening, TLS config, zone trust levels |
| `docs/deployment.md` | Install script options, what scripts/install.sh does, post-install setup, troubleshooting |
| `docs/config.md` | JSON schema for each subsystem config (dnsmasq, nginx, wireguard, cert types) |
| `docs/api.md` | REST API endpoint reference, request/response contracts, route patterns |
| `docs/hoover.md` | Custom frontend framework API reference |
| `docs/state-model.md` | Per-subsystem state schema, the `versions`/`tick` two-layer diff, and the pending-changes model |
| `docs/overview.md` | Subsystem summaries, tech stack, complete project directory tree |