vacuum-wall/AGENTS.md

Vacuum Wall — Agent Instructions

What This Is

SSL proxy / firewall appliance. Python 3.13+ Flask SPA behind nginx reverse proxy. Deploys on Debian 13 (trixie). Serves from repo root by default.

Architecture

Client ──→ nginx (TLS; basic auth on basic-authed proxy domains only) ──→ Flask (127.0.0.1:9090)
Flask  ──→ daemon/client.py (Unix socket) ──→ vacuum-walld (aiohttp, daemon.sock)
vacuum-walld ──→ daemon/handlers/*.py ──→ sudo <cmd> ──→ system service

Blueprints are thin proxies — they never call lib/ directly. All operations flow through the daemon client over a Unix socket.

Authentication: the management UI gets no nginx-level auth_basic — the mgmt server block's location / is a bare proxy and /ws is auth_basic off. Management auth is the Flask-layer JWT middleware (POST /api/auth/login → Authorization: Bearer <token>; public paths: static files, /vendor/, auth endpoints) plus the daemon WS handshake (raw JWT as the Sec-WebSocket-Protocol subprotocol). Basic auth (.htpasswd) renders only for proxy domains whose config/nginx/config.json has an auth block — never for the management domain (see docs/security.md, "Management Interface").

Two-User Model with Shared Group

• vacuum-walld (daemon user): runs the privileged background daemon with NOPASSWD sudo whitelist (/etc/sudoers.d/vacuum-walld). Owns socket. Primary group is the WebUI user's primary group. Daemon user name is derived: USER_NAME + d.
• WebUI user (default: repo owner in --dev mode): runs the Flask process with zero sudo access. Communicates with the daemon via Unix socket.
• Shared group: both users share the WebUI user's primary group. Socket is vacuum-walld:<group> with mode 0660.

Code Layout

• webui/server.py — Flask app entry point. Only file that creates the app. SPA root route (/) serves index.html (no templating). All other paths return 404.
• webui/api/*.py — Flask blueprints, one per subsystem. Routes prefix /api/<subsystem>/. All call daemon.client instead of lib/ directly.
• webui/api/common.py — Shared _ok() / _error() response helpers used by all blueprints.
• daemon/server.py — aiohttp server, route registry, batch routing, WebSocket broadcast, state refresh, periodic polling.
• daemon/client.py — Sync HTTP client over Unix socket using requests_unixsocket.Session.
• daemon/iface.py — Single source of truth for all daemon API endpoints. Every endpoint is a frozen (method, path) tuple. Renaming here auto-updates both server registry and client calls.
• daemon/handlers/*.py — Privileged operation handlers. All mutating sudo calls live here.
• daemon/collectors/ — Per-subsystem state collectors (7 modules: firewall, dnsmasq, nginx, acme, wireguard, networkd, system). Read-only sudo queries that populate lib.state. Imported for their registration side-effect; daemon/server.py imports the package before the first populate().
• lib/state.py — In-memory state store with per-subsystem collectors. Populated at daemon startup, refreshed on mutation/poll. Backs the WebSocket push stream: get_snapshot() (full state on WS connect), poll() two-layer diff (structural versions broadcast vs volatile-only tick broadcast, per-subsystem, each carrying the full subsystem data), register_volatile(subsystem, keys) to mark volatile fields, get_versions()/bump(). Per-subsystem poll intervals via _DEFAULT_POLL_INTERVALS (system 1s, firewall 30s, wireguard/dnsmasq/networkd 10s, nginx 60s, acme 300s).
• lib/common.py — Shared utilities: run(), run_proc(), load_json(), save_json(), deep_merge(), ensure_dirs(), config_hash(), validate_interface_name(), plus the apply-bookkeeping helpers stamp_applied() / strip_apply_meta() / compute_pending() / deep_diff() / revert_to_applied() (the _last_applied_hash / _last_applied_config keys every config-backed subsystem uses for pending-change detection and cancel-all).
• lib/logging.py — Logging setup used by both webui and daemon. Reads VACUUM_WALL_LOG_LEVEL.
• lib/sync.py — Cross-subsystem sync event bus (in-process pub/sub); handlers emit events on mutation and subscribers refresh affected subsystems.
• lib/system_import.py — Startup system-config import/reconcile: parses native config sources and merges them into the declarative JSON on daemon start.
• lib/bootstrap.py — Daemon-startup filesystem bootstrap, run after system_import.import_all() (which must see absent config files to adopt live state on first start) and before the first state collection: creates the runtime config/+data/ directories and persists the one-shot nginx legacy-format migration. Never creates config files (reads stay pure; files appear on first save_config).
• lib/schema.py — TypedDict state schemas for the per-subsystem state payloads.
• lib/*.py — Backend modules (parsing, config, shared logic). Full type hints and __all__ exports. No sudo calls.
• vendor/ — Vendored scripts and JS libraries (acme.sh, htm).
• data/ — Runtime artifacts (generated .confs, .htpasswd, ACME certs, firewall backup, dnsmasq fragments).
• config/<subsystem>/config.json — Declarative JSON configs (source of truth).
• system/ — System file templates. systemd/ (units installed to /etc/systemd/system/), sudoers.d/, nginx/.

Project uses .venv. Install deps with pip install -e . (from pyproject.toml). __init__.py files in webui/, lib/, and daemon/ are intentionally empty.

Frontend (hoover)

Custom reactive SPA framework at webui/static/hoover/. See docs/hoover.md for full API reference.

Conventions:

• All imports from /static/hoover/index.js (barrel export).
• Pages in webui/static/pages/ export definePage({ init, subscribe, load, render }).
• Bootstrap: webui/static/app.js mounts two render roots (#sidebar, #main), then connect() for WS.
• h() builds VNodes with on:click prefix. html tag (htm) templates use camelCase onClick (adapter translates).
• State always has loading, refreshing, error plus data. load() receives (state, abortController, entry).
• openModal + formModal for dialogs; apiSubmit() for form submission.
• No build step — ES modules served raw. Cache controlled via HTTP headers. For the management domain, nginx serves /static/ directly from webui/static/ (generated location /static/ alias with no-cache + ETag revalidation); Flask's static route is the dev-mode fallback.

Daemon Endpoints

• Unix socket at data/daemon.sock (configurable via VACUUM_WALLD_SOCKET)
• WebSocket at 127.0.0.1:9091 (configurable via VACUUM_WALLD_WS_PORT) for real-time state streaming: full snapshot on connect, then per-subsystem data-carrying versions (structural) / tick (volatile-only) deltas. The client patches reactive models in place via modelSet() — no HTTP round-trip for auto-refresh.
• Periodic polling per subsystem via lib.state._DEFAULT_POLL_INTERVALS, overridable with VACUUM_WALL_POLL_INTERVALS env var (format subsystem:seconds,subsystem:seconds)
• Start as python -m daemon.server or via the vacuum-walld console script

Environment Variables

• VACUUM_WALL_DEV — dev mode flag; disables aggressive static asset caching
• VACUUM_WALL_LOG_LEVEL — log level (default INFO)
• VACUUM_WALLD_SOCKET — override daemon socket path (default data/daemon.sock)
• VACUUM_WALLD_WS_PORT — override WebSocket port (default 9091)
• VACUUM_WALL_POLL_INTERVALS — override poll intervals, e.g. firewall:60,wireguard:5
• VACUUM_WALL_EXTERNAL_IP_URL — custom URL for external IP detection (acme handler)
• VACUUM_WALL_SEED_BUILTIN_ADMIN — set to 0 to skip the last-resort builtin admin seed in get_db(). The seed only runs on a completely empty DB (no users); scripts/bootstrap_auth.py always sets this since bootstrap creates the operator user itself.

Local Dev

# Setup
python3 -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"
bash scripts/update-vendor.sh        # fetches acme.sh + htm.js

# Start (Flask only, binds 127.0.0.1:9090)
.venv/bin/python webui/server.py

Reload running Flask via SIGHUP (auto-reloads webui.* and lib.* modules, then SIGTERM restart).

Blueprint / Handler / lib Mapping

Blueprint	URL Prefix	Handler	lib Module
webui/api/firewall	/api/firewall/	daemon/handlers/firewall	lib.firewall
webui/api/dhcp	/api/dhcp/	daemon/handlers/dnsmasq	lib.dnsmasq
webui/api/proxy	/api/proxy/	daemon/handlers/nginx	lib.nginx
webui/api/certs	/api/certs/	daemon/handlers/acme	lib.acme
webui/api/wireguard	/api/wireguard/	daemon/handlers/wireguard	lib.wireguard
webui/api/network	/api/network/	daemon/handlers/network	lib.network
webui/api/logs	/api/logs/	daemon/handlers/logs	—
webui/api/status	/api/status/	daemon/handlers/status	—
webui/api/auth	/api/auth/	daemon/handlers/auth	lib.auth / lib.auth_users

Privileged Operations

daemon/handlers/*.py call sudo for everything that touches system services. Whitelist is system/sudoers.d/vacuum-walld.

acme.sh must never run as root — always as the service user via sudo -u.

Pattern for mutations: write JSON → render native config → sudo <cmd> to apply. Adding a new privileged command requires a sudoers entry and the daemon/handlers/ code.

Config reads are pure. Every lib/<subsystem>.get_config() is a side-effect-free read (returns in-memory defaults when the file is missing; nginx applies its legacy-format migration in memory). State collectors therefore never write to disk — filesystem setup (runtime dirs, one-shot nginx migration) happens once at daemon startup in lib/bootstrap.py (after system_import.import_all()).

Firewall interface-coverage invariant. Every network-managed interface (lo/wg* excluded) must be covered by a zone in config/firewall/config.json or listed under the top-level unmanaged key. The config is the source of truth for zone interfaces (an omitted interfaces key = empty list; no hands-off zones). Enforced at save time (POST/PATCH /firewall/config → 400) and apply time (POST /firewall/config/apply → 409, force: true overrides) via the pure lib.firewall.validate_coverage(). Live drift is advisory only (uncovered_interfaces state field). See docs/config.md.

API Response Contract

• Success: {"ok": true, "data": <value>} — _ok(data) (Flask) or ok(data) (aiohttp)
• Error: {"ok": false, "error": "msg"} — _error(msg, code) (Flask) or error(msg, code) (aiohttp)
• acme.issue() / acme.renew() raise RuntimeError on failure — API layer wraps in try/except
• HTTP codes: 400 bad request, 404 not found, 409 conflict, 500 internal failure

Deploy

scripts/install.sh is the single deploy script. Run as root. CLI flags take precedence over env vars. MGMT_PASS is strictly required — it is the SQLite DB password for the initial admin user (full rw on all subsystems; default username admin), not an nginx htpasswd. MGMT_DOMAIN auto-detected from hostname.

Service Start Order

firewalld → avahi-daemon → dnsmasq → vacuum-walld → vacuum-wall

Lint and Tests

Linter / formatter: Ruff (ruff check + ruff format). Config in pyproject.toml.

Tests: pytest in tests/ (28 Python + 9 JS test files). All subprocess calls are mocked — no system services required.

.venv/bin/ruff check lib/ webui/ tests/   # lint
.venv/bin/ruff format lib/ webui/ tests/   # format
.venv/bin/python -m pytest tests/ -v       # test

Docs

docs/ contains the authoritative reference for each subsystem. Read the relevant doc before reasoning about a subsystem.

Doc	Contents
docs/architecture.md	Request flow, subsystem communication, two-user model, zone model, state management
docs/security.md	Privilege model, sudo whitelist, systemd hardening, TLS config, zone trust levels
docs/deployment.md	Install script options, what scripts/install.sh does, post-install setup, troubleshooting
docs/config.md	JSON schema for each subsystem config (dnsmasq, nginx, wireguard, cert types)
docs/api.md	REST API endpoint reference, request/response contracts, route patterns
docs/hoover.md	Custom frontend framework API reference
docs/state-model.md	Per-subsystem state schema, the versions/tick two-layer diff, and the pending-changes model
docs/overview.md	Subsystem summaries, tech stack, complete project directory tree