mteehan/vacuum-wall
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: full refresh per DOCSPLAN (auth subsystem, backends model, access classes, sudo table, state-model mechanics) + 3 stale docstrings

Browse Source
This commit is contained in:
Mike Teehan
2026-09-05 16:34:57 +00:00
parent 78fcb01877
commit b503a6dcf0
13 changed files with 1468 additions and 543 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/overview.md
+114 -45
View File
@@ -6,13 +6,17 @@ Vacuum Wall is a zone-based firewall appliance with a built-in SSL reverse proxy
## Architecture Overview
Vacuum Wall is built around five integrated subsystems managed through a two-layer architecture: a non-privileged Flask web UI and a privileged background daemon (`vacuum-walld`). The web UI communicates with the daemon via a Unix socket. The daemon handles all privileged operations (sudo) for the subsystems: the traffic plane uses firewalld with its nftables backend (zone-based policies, source NAT, destination NAT); the DNS/DHCP plane serves private subnets via dnsmasq; the network plane uses systemd-networkd for static IP management; the proxy plane runs nginx with automatic ACME certificates through acme.sh; and the VPN plane uses WireGuard (wg-quick) for encrypted tunnel management. All subsystems are configured and monitored through the Flask web UI, which is itself proxied through nginx (TLS termination only — management authentication is a Flask-layer JWT, not nginx basic auth; individual proxy domains may optionally configure their own basic auth).
Vacuum Wall is built around six integrated subsystems managed through a two-layer architecture: a non-privileged Flask web UI and a privileged background daemon (`vacuum-walld`). The web UI communicates with the daemon via a Unix socket; the daemon also streams real-time state over a local WebSocket (127.0.0.1:9091) — a full `snapshot` on connect, then per-subsystem `versions` (structural) and `tick` (volatile-only) deltas — so the UI auto-refreshes without HTTP polling. The daemon handles all privileged operations (sudo) for the subsystems: the traffic plane uses firewalld with its nftables backend (zone-based policies, source NAT, destination NAT); the DNS/DHCP plane serves private subnets via dnsmasq; the network plane uses systemd-networkd for static IP management; the proxy plane runs nginx with automatic ACME certificates through acme.sh; the VPN plane uses WireGuard (wg-quick) for encrypted tunnel management; and the authentication subsystem manages users, passkeys, and JWT sessions. Certificate management is tracked as a standalone state subsystem with its own API. In total, `lib/state.py` tracks 7 state subsystems. All subsystems are configured and monitored through the Flask web UI, which is itself proxied through nginx (TLS termination only — management authentication is a Flask-layer JWT, not nginx basic auth; individual proxy domains may optionally configure their own basic auth).
## Subsystems
### Firewall
The firewall uses firewalld's zone model for traffic control. Network interfaces are assigned to zones such as external, internal, VPN, and trusted. Rules and services define which traffic is allowed between zones. Source NAT (masquerade) enables RFC 1918 networks to reach the internet through the external interface. Destination NAT rules provide port forwarding, exposing internal services to external networks on configurable ports.
The firewall uses firewalld's zone model for traffic control. Network interfaces are assigned to zones such as external, internal, and trusted, plus a per-access-class `vpn-<class>` zone for each WireGuard access class (managed by the WireGuard sync). Zones carry an optional per-zone `target` (accept/drop/reject), and rules express fine-grained policies via services, port rules, and rich rules. Source NAT (masquerade) enables RFC 1918 networks to reach the internet through the external interface. Destination NAT rules provide port forwarding, exposing internal services to external networks on configurable ports.
**Interface-coverage invariant.** Every network-managed interface (`lo`/`wg*` excluded) must be covered by a zone in the firewall config or declared in the top-level `unmanaged` list. The invariant is enforced at save time (400) and at apply time (409; `{"force": true}` overrides); live drift is advisory only and surfaced as `uncovered_interfaces` in state.
**Pending-changes model.** Edits saved to a config are not applied until the operator applies them. Each subsystem exposes `pending_changes` plus a `pending_diff` of the changed fields, aggregated at `GET /api/status/pending`. `POST /api/status/apply-all` applies pending changes in dependency order (networkd → firewall → wireguard → dnsmasq → nginx); `POST /api/status/cancel-all` reverts all pending edits to the last-applied config.
### DHCP/DNS
@@ -20,26 +24,38 @@ dnsmasq serves as both the DHCP server and local DNS resolver. It is configured
### SSL Proxy
The nginx reverse proxy handles HTTPS termination for user-defined domains, with certificates automatically provisioned and renewed via acme.sh and an ACME provider (Let's Encrypt by default). Each proxy domain is configured with an HTTP-to-HTTPS redirect, modern TLS settings, and a configurable backend target. New proxy domains are added through the web UI, and the configuration is applied without manual intervention.
The nginx reverse proxy handles HTTPS termination for user-defined domains, with certificates automatically provisioned and renewed via acme.sh and the configured ACME provider (the CA is config-driven; the code default is Let's Encrypt). The configuration is a three-part model: named `backends`, `domains` that reference them, and a global `ssl` settings block. Each domain's paths resolve against its named backend's path table, and a builtin `webui` backend serves the management interface (Flask on 127.0.0.1:9090 plus the WebSocket on 127.0.0.1:9091). Backends are managed through the web UI (list, add, update, remove). Proxy domains may additionally gate paths with per-domain basic auth via a generated `.htpasswd` file — never on the management domain, which relies on the Flask-layer JWT. New proxy domains are added through the web UI, and the configuration is applied without manual intervention.
### Network (systemd-networkd)
The networkd subsystem manages static IP configuration for network interfaces via systemd-networkd. It renders declarative JSON configuration into per-interface `.network` INI files (`50-<name>.network`), supporting static addresses, routes, DNS, DHCP clients, link settings, and all `[Address]`, `[Route]`, `[DHCPv4]`, `[DHCPv6]`, and `[Link]` section keys. When the full apply runs, public DNS servers from networkd configs are auto-synced to dnsmasq's upstream resolvers. Helper endpoints can infer candidate DHCP ranges from static IPs and suggest firewalld zone assignments based on interface role.
The networkd subsystem manages static IP configuration for network interfaces via systemd-networkd. It renders declarative JSON configuration into per-interface `.network` INI files (`99-<name>.network`), supporting static addresses, routes, DNS, DHCP clients, link settings, and all `[Address]`, `[Route]`, `[DHCPv4]`, `[DHCPv6]`, and `[Link]` section keys. When the handler applies an interface, it removes lower-priority conflicting `.network` files from the system directory. When the full apply runs, public DNS servers from networkd configs are auto-synced to dnsmasq's upstream resolvers. Helper endpoints can infer candidate DHCP ranges from static IPs and suggest firewalld zone assignments based on interface role.
### WireGuard
WireGuard support provides server-side VPN tunnel management. Peers are added through the web UI, with the system generating client configuration files that can be downloaded and applied on remote devices. The dashboard displays active connections and transfer statistics for each peer, allowing operators to monitor tunnel health and usage.
WireGuard support provides server-side VPN tunnel management. Tunnels are organized into **access classes**: each class owns a `wg-<class>` interface, a `vpn-<class>` firewall zone, a dedicated subnet, listen port, and keypair, plus a `lan_access` flag controlling whether its peers can reach the LAN. Two classes exist by default (`full`, with LAN access, and `internet`, without). Classes are managed through CRUD endpoints (add, update, delete, reorder, generate keys). Peers are assigned to a class and added through the web UI, with the system generating client configuration files that can be downloaded and applied on remote devices. The dashboard displays active connections and transfer statistics for each peer, allowing operators to monitor tunnel health and usage.
### Authentication
Authentication is a first-class subsystem. Users, per-subsystem read/rw permissions, Argon2id password hashes, and optional passkeys (WebAuthn/FIDO2) are stored in a SQLite database (`data/auth.db`), reached through an abstract database layer that never exposes raw SQL. Sessions use JWT access + refresh tokens: each user holds their own HS256 signing secret, and revoked tokens are blacklisted by `jti`. A builtin `admin` user is seeded at bootstrap. The subsystem exposes `/api/auth/*` endpoints and the login, users, and passkeys pages.
### Certificates (ACME)
Certificate management is a standalone state subsystem with its own API (`/api/certs/*`). acme.sh issues and renews certificates for proxy domains against the configured CA provider; self-signed certificates can be generated for domains without an ACME account, and ACME accounts can be registered or deactivated. A systemd timer runs periodic renewals, and certificate state (issuance, expiry) is collected like any other subsystem.
## Tech Stack
- Debian 13 (trixie) target platform
- Python 3.13+, Flask 3.x for web management
- aiohttp (daemon server) + requests-unixsocket (Unix-socket client)
- firewalld (nftables backend)
- systemd-networkd (ip-lladdr, networkctl)
- nginx 1.26+
- systemd-networkd (networkctl)
- nginx
- dnsmasq
- WireGuard tools (wireguard-tools)
- acme.sh for ACME certificate management (Let's Encrypt by default)
- acme.sh for ACME certificate management (CA provider config-driven; code default Let's Encrypt)
- SQLite (auth database)
- PyJWT (JWT sessions), argon2-cffi (Argon2id password hashing), webauthn (passkeys), passlib (htpasswd only)
- htm.js (vendored JS tagged-template HTML adapter)
## Quick Start
@@ -58,65 +74,107 @@ After installation, access the management interface at `https://<hostname>.local
## Project Structure
```
├── README.md # Project overview
├── AGENTS.md # Agent instructions
├── .gitignore
├── pyproject.toml # Project metadata + dependencies
├── scripts/ # Utility scripts
│ ├── install.sh # Deployment script (renders Jinja2 templates)
│ └── update-vendor.sh # Download vendored libraries (acme.sh, htm)
├── pyproject.toml # Project metadata + dependencies
├── .venv/ # Python virtual environment
│ ├── update-vendor.sh # Download vendored libraries (acme.sh, htm)
│ ├── bootstrap_auth.py # Auth DB bootstrap (creates the operator user)
│ └── restart-services.sh # Restart installed system services
├── config/ # Declarative JSON configuration (source of truth)
│ ├── firewall/ # Firewall zone & rule config
│ ├── dnsmasq/ # DHCP/DNS config
│ ├── network/ # systemd-networkd per-interface config
│ ├── firewall/ # Firewall zone & rule config
│ ├── nginx/ # Proxy domain & SSL config
│ ├── wireguard/ # VPN interface & peer config
│ └── acme/ # ACME account settings (email, CA provider)
│ ├── nginx/ # Proxy backend, domain & SSL config
│ ├── wireguard/ # VPN access-class, interface & peer config
│ ├── acme/ # ACME account settings (email, CA provider)
│ └── auth/ # Authentication settings (JWT, WebAuthn)
├── data/ # Runtime artifacts & generated files
│ ├── auth.db # SQLite auth database (users, passkeys)
│ ├── certs/ # Management-domain TLS keypair
│ ├── daemon.sock # Daemon Unix socket
│ ├── nginx/sites-enabled/ # Generated server blocks
│ ├── nginx/.htpasswd # Basic-auth entries for proxy domains
│ ├── dnsmasq/fragments/ # User config fragments
│ ├── acme/ # ACME certificates
│ ├── firewall/ # Pre-apply recovery snapshot
│ ├── logs/ # Application logs
│ ├── networkd/ # Generated 50-<name>.network files
│ └── wireguard/ # Generated WireGuard configs
│ ├── acme/ # acme.sh home: certs, account, webroot (www/)
│ ├── firewall/rules.json # Pre-apply recovery snapshot
│ ├── networkd/ # Generated 99-<name>.network files
│ ├── wireguard/ # Generated WireGuard configs
│ └── logs/ # Application logs
├── daemon/ # Privileged background daemon
│ ├── server.py # aiohttp server, cache, batch routing, handler registry
│ ├── server.py # aiohttp server: endpoint registry (daemon/iface.py), batch routing, WebSocket broadcast (snapshot/versions/tick), state refresh, per-subsystem polling
│ ├── client.py # Sync HTTP client over Unix socket
│ ├── iface.py # Single source of truth for daemon API endpoints
│ ├── __main__.py # Module entry point (python -m daemon.server)
│ ├── handlers/ # Privileged operation handlers (all sudo calls)
│ │ └── network.py # networkd handler (generate + apply)
├── system/ # System file templates (all Jinja2)
│ │ ├── firewall.py # Zone/rich-rule CRUD + apply
│ │ ├── dnsmasq.py # DHCP/DNS config + apply
│ │ ├── nginx.py # Proxy domain/backend + SSL apply
│ │ ├── network.py # networkd handler (generate + apply)
│ │ ├── wireguard.py # Access-class/peer CRUD + tunnel control
│ │ ├── acme.py # Certificate issue/renew/self-signed, account
│ │ ├── auth.py # User/passkey management
│ │ ├── logs.py # Log streaming
│ │ ├── status.py # Pending/apply-all/cancel-all
│ │ ├── system.py # System info & metrics
│ │ └── common.py # Shared handler helpers (sync emit + refresh)
│ └── collectors/ # Read-only per-subsystem state collectors
│ ├── firewall.py # firewall collector
│ ├── dnsmasq.py # dnsmasq collector
│ ├── networkd.py # networkd collector
│ ├── nginx.py # nginx collector
│ ├── wireguard.py # wireguard collector
│ ├── acme.py # acme collector
│ └── system.py # system collector
├── system/ # System file templates (mostly Jinja2)
│ ├── systemd/ # Service and timer unit files
│ │ ├── vacuum-wall.service # Web UI service (rendered at install)
│ │ ├── vacuum-wall-acme.service # Certificate renewal (rendered at install)
│ │ ├── vacuum-wall-acme.timer # Renewal schedule
│ │ └── vacuum-walld.service # Privileged daemon (rendered at install)
│ ├── sudoers.d/ # Sudo whitelist (rendered at install)
│ ├── tmpfiles.d/ # tmpfiles.d spec (installed verbatim)
│ ├── tmpfiles.d/ # tmpfiles.d spec (installed verbatim, not Jinja)
│ ├── nginx/ # Nginx config templates (rendered at runtime)
│ ├── dnsmasq.conf # Dnsmasq template (rendered at runtime)
│ └── wireguard*.conf # WireGuard templates (rendered at runtime)
│ ├── wireguard.conf # WireGuard server template (rendered at runtime)
│ ├── wireguard-client.conf# WireGuard client template (rendered at runtime)
│ ├── acme-deploy.py # ACME deploy hook (installed verbatim, not Jinja)
│ └── acme-deploy.sh # ACME deploy wrapper (installed verbatim, not Jinja)
├── lib/ # Subsystem abstraction layer
│ ├── common.py # Shared utilities (run, run_proc, load_json, save_json, deep_merge, ensure_dirs, get_interface_ip)
│ ├── common.py # Shared utilities (run, run_proc, load_json, save_json, deep_merge, ensure_dirs, get_interface_ip, config_hash, stamp_applied, strip_apply_meta, compute_pending, deep_diff, revert_to_applied, validate_interface_name)
│ ├── logging.py # Logging setup
│ ├── firewall.py # firewalld bindings
│ ├── network.py # systemd-networkd rendering & parsing
│ ├── dnsmasq.py # DHCP/DNS configuration
│ ├── nginx.py # Reverse proxy configuration
│ ├── state.py # State collector (uses lib.network.parse_networkctl_status)
│ ├── nginx.py # Reverse proxy configuration (backends model)
│ ├── state.py # In-memory state store (per-subsystem data, version counters, two-layer versions/tick diff, poll intervals, volatile registration); collectors live in daemon/collectors/
│ ├── sync.py # Cross-subsystem event bus
│ ├── acme.py # Certificate management (ACME helpers)
│ ├── wireguard.py # VPN tunnel and peer management
│ └── system_import.py # Startup reconciler (imports live system configs into JSON)
│ ├── system_import.py # Startup reconciler (imports live system configs into JSON)
│ ├── bootstrap.py # Daemon-startup filesystem bootstrap
│ ├── schema.py # TypedDict state schemas
│ ├── auth.py # JWT access+refresh tokens, per-user HS256 secrets, jti blacklist
│ ├── auth_users.py # Multi-user management, per-subsystem read/rw permissions, builtin admin
│ ├── password.py # Argon2id password hashing
│ ├── webauthn.py # Passkey (FIDO2/WebAuthn) support
│ ├── db.py # Abstract database layer (opaque query IDs)
│ └── db_sqlite.py # SQLite backend (data/auth.db)
├── webui/ # Flask web application
│ ├── server.py # Application entry point
│ ├── api/ # REST API route modules (blueprints)
│ │ ├── common.py # Shared API response helpers (_ok, _error)
│ │ ├── firewall.py # Firewall API
│ │ ├── dhcp.py # DHCP/DNS API
│ │ ├── proxy.py # Nginx proxy API
│ │ ├── proxy.py # Nginx proxy API (domains + backends)
│ │ ├── certs.py # Certificate API
│ │ ├── wireguard.py # WireGuard API
│ │ ├── network.py # Networkd API
│ │ └── logs.py # Logs API
│ │ ├── logs.py # Logs API
│ │ ├── auth.py # Authentication API
│ │ └── status.py # Status API (pending/apply-all/cancel-all)
│ └── static/ # SPA (index.html, app.js, style.css)
│ ├── hoover/ # Hoover SPA framework (VDOM, reactivity, router, components)
│ │ ├── index.js # Barrel export of all public APIs
@@ -128,22 +186,32 @@ After installation, access the management interface at `https://<hostname>.local
│ │ ├── websocket.js
│ │ ├── api.js
│ │ ├── helpers.js
│ │ └── components/ # Layout, data display, modal, toast
│ └── pages/ # Page modules (each defines a route via definePage)
│ │ ├── html.js # htm.js tag adapter
│ │ ├── model.js # Reactive model store
│ │ ├── auth_model.js# Auth session model
│ │ ├── dirty.js # Dirty-state tracking
│ │ ├── schema.js # Schema validation helpers
│ │ └── components/ # applyconfirm, auth, data, layout, modal, qr, toast
│ └── pages/ # 15 page modules (each defines a route via definePage):
│ # dashboard, zones, rules, nat, interfaces, dhcp,
│ # proxy, backends, certs, wireguard, logs, login,
│ # users, passkeys, notfound
├── vendor/ # Vendored scripts and JS libraries
│ ├── acme.sh # ACME certificate client
│ └── htm.js # JS tagged-template HTML adapter
├── docs/ # Documentation
│ ├── overview.md # This file
│ ├── deployment.md
│ ├── api.md
│ ├── security.md
│ ├── architecture.md
│ ├── config.md
│ └── hoover.md # Hoover SPA framework
└── scripts/ # Utility scripts
├── install.sh # Deployment script (renders Jinja2 templates)
└── update-vendor.sh # Download vendored libraries (acme.sh, htm)
│ ├── htm.js # JS tagged-template HTML adapter
│ └── qrcode-svg-1.1.0.js # QR code generation (SVG)
├── tests/ # Test suites
│ ├── test_*.py # 28 Python modules (pytest; subprocess calls mocked)
│ └── test-*.js # 9 JS test modules (hoover framework)
└── docs/ # Documentation
├── overview.md # This file
├── deployment.md
├── api.md
├── security.md
├── architecture.md
├── config.md
├── state-model.md # State schema, versions/tick diff, pending-changes model
└── hoover.md # Hoover SPA framework
```
## Documentation
@@ -153,4 +221,5 @@ After installation, access the management interface at `https://<hostname>.local
- [Security Model](security.md) - Privilege model and sudo whitelist
- [Architecture](architecture.md) - Detailed subsystem design
- [Configuration](config.md) - Config file formats and locations
- [State Model](state-model.md) - State schema, versions/tick diff, pending-changes
- [Hoover Framework](hoover.md) - Frontend SPA framework reference