# Architecture

## Request Flow

The following describes the path a request takes from an external client to a backend service and back:

### Proxied Service (e.g., `app.example.com`)

1. An external client sends an HTTP request to `app.example.com`.
2. The request arrives at the Vacuum Wall host's WAN interface, assigned to the `external` firewalld zone. A firewall rule allows inbound traffic on port 443 (HTTPS).
3. nginx, listening on port 443, terminates the TLS connection using the domain's certificate.
4. nginx evaluates the `server_name` against the configured server blocks. The matching block is generated from the domain entry in `config/nginx/config.json`.
5. If the domain has an `auth` block, nginx applies HTTP Basic authentication before proxying. Auth is resolved in the order domain → backend (the effective `auth` is the domain's own, or the referenced backend's if the domain has none), and per-path behavior follows the resolved auth config. Credentials are checked against the generated `data/nginx/.htpasswd` file; unauthenticated requests receive a 401 with a `WWW-Authenticate` challenge. Domains without an `auth` block skip this step entirely.
6. The request is forwarded to the backend service (e.g., `192.168.2.50:8080`) via a `proxy_pass` directive (per-path upstreams resolved from the backend's `paths` config).
7. The backend service processes the request and returns an HTTP response.
8. nginx adds security headers (`X-Content-Type-Options`, `X-Frame-Options`, HSTS, etc.) to the response.
9. nginx encrypts the response with TLS and sends it back to the client through the WAN interface.

For HTTP requests (port 80), nginx returns a 301 redirect to the HTTPS equivalent before any proxying occurs.

### Management WebUI Access (e.g., `<hostname>.local`)

1. A client sends an HTTPS request to the management domain.
2. nginx terminates TLS and proxies the request to `127.0.0.1:9090` where the Flask WebUI is listening. No nginx-level authentication is applied.
3. Flask validates the JWT from the `Authorization: Bearer <token>` header together with the `X-Session-Id` header (both are required; the session ID must match the token's `session_id` claim), checks the token against the SQLite blacklist (`data/auth.db`), and verifies per-subsystem permissions before processing the request. Public endpoints (login, token refresh, WebAuthn authenticate) are exempt from validation.
4. The Flask application communicates with the `vacuum-walld` daemon via a Unix socket (`data/daemon.sock`) for any privileged operations.
5. The daemon executes the privileged commands via the sudo whitelist and returns structured results.
6. Flask renders an HTML or JSON response, which nginx returns to the client over the encrypted connection.

Because Flask binds only to `127.0.0.1`, it is unreachable directly from any external interface. The nginx reverse proxy is the sole entry point.

## Subsystem Communication

The following diagram summarizes how the Flask WebUI communicates with each managed subsystem:

```
External Client ──→ nginx (SSL termination; auth_basic only on proxy domains with an `auth` block) ──→ Flask WebUI (127.0.0.1:9090, JWT + X-Session-Id + permission check)
Flask WebUI ──→ daemon/client.py (path resolution, Unix socket) ──→ vacuum-walld (aiohttp server)
Flask WebUI ──→ lib/db.py (abstract DB interface) ──→ SQLite (data/auth.db)
vacuum-walld ──→ daemon/handlers/auth.py ──→ lib/auth.py ──→ JWT operations
vacuum-walld ──→ daemon/handlers/firewall.py ──→ sudo firewall-cmd ──→ firewalld / D-Bus ──→ nftables
vacuum-walld ──→ daemon/handlers/nginx.py ──→ render temps in /run/vacuum-wall ──→ sudo cp to /etc/nginx/ ──→ sudo nginx -t && sudo nginx -s reload (SIGHUP)
vacuum-walld ──→ daemon/handlers/dnsmasq.py ──→ render config ──→ /run/vacuum-wall/dnsmasq.tmp ──→ sudo cp to /etc/dnsmasq.d/vacuum-wall.conf ──→ sudo systemctl restart dnsmasq
vacuum-walld ──→ daemon/handlers/acme.py ──→ acme.sh (subprocess) ──→ on issue/renew success: deploy hook (sudo nginx -t && sudo nginx -s reload) ──→ nginx
vacuum-walld ──→ daemon/handlers/wireguard.py ──→ render per-class config (wg-<class>) ──→ /run/vacuum-wall/<ifname>.conf.tmp (0600) ──→ sudo cp to /etc/wireguard/<ifname>.conf ──→ sudo wg-quick up <ifname>
vacuum-walld ──→ daemon/handlers/network.py ──→ render 99-<name>.network ──→ sudo cp to /etc/systemd/network/ ──→ sudo networkctl reload + sudo networkctl reconfigure <iface>
vacuum-walld ──→ daemon/handlers/logs.py ──→ sudo journalctl ──→ systemd journal
vacuum-walld ──→ daemon/handlers/status.py ──→ cross-subsystem apply-all (networkd→firewall→wireguard→dnsmasq→nginx) + cancel-all (revert to last applied)
vacuum-walld ──→ daemon/handlers/system.py ──→ pre-collected /proc metrics (state store)
```

Notes on the diagram:

- The ACME flow terminates at nginx: acme.sh stores certs on disk and the `deploy` step fires the deploy hook (`system/acme-deploy.sh`, installed to `$ACME_HOME/deploy/` by the install script) only after a successful issue or renewal; the hook runs `sudo nginx -t && sudo nginx -s reload`. A Python hook variant (`system/acme-deploy.py`) that instead calls the daemon's `POST /nginx/reload` endpoint exists in the tree but is not the one the install script installs.
- WireGuard multi-interface mode: the config defines `access_classes`; each class with peers gets its own interface `wg-<class>`, rendered to `/etc/wireguard/wg-<class>.conf`. Legacy single-interface mode renders `/etc/wireguard/wg0.conf`.

### Two-User Model with Shared Group

Vacuum Wall uses two distinct system users bridged by a shared group:

- **`vacuum-walld`** (daemon user): Runs the privileged background daemon. Holds the NOPASSWD sudo whitelist for all system-level commands. Runs with `NoNewPrivileges=yes` (satisfiable since sudo is called directly by the daemon process). Its systemd unit declares `RuntimeDirectory=vacuum-wall nginx` (pre-creates `/run/vacuum-wall` and `/run/nginx` before namespace setup) and `LogsDirectory=vacuum-wall` (`/var/log/vacuum-wall`; the management unit declares the same `LogsDirectory`).
- **WebUI user** (default: repo owner in `--dev` mode): Runs the Flask web serving process. Has **zero** sudo access. Communicates with the daemon via a Unix socket at `data/daemon.sock`. Runs with `NoNewPrivileges=yes`.
- **Shared group**: Both users share the WebUI user's primary group. The daemon socket is owned by `vacuum-walld:<group>` with mode `0660`, allowing the web UI user to connect via group permission. In **production**, the project directory is owned by the **daemon user** with group read+write (`g+rwX`) and the setgid bit on all subdirectories, so the WebUI user can read configs and shared files via the shared group. In **`--dev` mode only**, the project directory stays owned by the repo owner (the WebUI user).

This design isolates privilege escalation entirely within the daemon, so a compromised Flask process cannot invoke sudo directly. All *mutating* privileged operations live in `daemon/handlers/*.py`, but a few `lib/` code paths still execute sudo and are only ever called from within the daemon process:

- `lib/common.get_interface_ip` — `sudo ip -o addr show <iface>` (used by sync subscribers and handlers to backfill gateway addresses)
- `lib/system_import.import_firewall` — `sudo firewall-cmd --list-all-zones` (startup import only)
- `lib/nginx.test_config` — `sudo nginx -t` (imported live by `daemon/handlers/acme.py` for ACME pre-flight checks)
- Legacy sudo code in `lib/nginx.py` (install/reload helpers) and `lib/wireguard.py` (legacy apply/down paths)

**Dev mode variant**: When `scripts/install.sh --dev` is used, the repo owner (e.g., `wall`) becomes the WebUI user. The project directory remains owned by the repo owner, preserving git operations and code editing. The daemon user (`vacuum-walld`) has the repo owner's primary group as its own primary group, granting read access to project files. All subdirectories carry the setgid bit (`g+s`) so new files inherit the group regardless of the creator's primary group.

The `lib/` modules auto-discover the project root at runtime via `Path(__file__).resolve().parent.parent`. This works because `scripts/install.sh` performs an editable pip install (`pip install -e .`), keeping module files in the project directory rather than copying them to `site-packages/`.

## JWT Token Model

Vacuum Wall uses JWT-based authentication with access/refresh token rotation. Tokens are stored in browser `sessionStorage` and injected as `Authorization: Bearer <token>` headers. The API never reads cookies — authentication is header-only.

| Token | Lifetime | Storage | Purpose |
|---|---|---|---|
| Access | 5 min on fresh install (config-driven; code fallback 900 s) | sessionStorage / memory | API auth, permission checks |
| Refresh | 7 days | sessionStorage | Token rotation, new access tokens |

Every JWT payload contains `sub` (username), `exp` (expiry), `iat` (issued at), `jti` (unique identifier), and `type` (`"access"` or `"refresh"`). **Access tokens additionally carry `permissions` (per-subsystem permissions) and `session_id` (session binding — always present; a fresh ID is generated if the caller does not supply one). Refresh tokens carry only `session_id`, and only when a session was bound at login — they never carry `permissions`.** The Flask middleware relies on both: it reads `permissions` from the access token and cross-checks the `X-Session-Id` request header against the token's `session_id` claim.

Each user has a unique signing secret stored in the `users.jwt_secret` database column (generated as a 32-byte base64url token via `secrets.token_urlsafe(32)`). This per-user secret model means tokens signed for one user cannot be validated as another user's tokens. Both Flask and daemon processes validate tokens by extracting `sub` from the unverified payload, looking up the user's secret, and verifying the signature with that secret. Expired and blacklisted tokens are rejected against the SQLite `token_blacklist` table (via `data/auth.db`).

Token auto-refresh occurs before expiry. On logout or password change, tokens are blacklisted in the SQLite `token_blacklist` table to prevent reuse. Blacklist cleanup is **probabilistic, not per-refresh**: `blacklist_token()` purges expired entries with a 2% chance on each call, and the daemon's poll loop additionally runs `blacklist_expired()` at most every 60 seconds (coordinated across all poll loops via a shared lock).

## Permission Model

Each user has per-subsystem permissions with two levels:

- **`"read"`** — `GET /api/<subsystem>/*` allowed; `POST`/`PATCH`/`DELETE` rejected with 403
- **`"rw"`** — all HTTP methods allowed for the subsystem

Flask `before_request` middleware enforces permissions by extracting the subsystem name from the blueprint route prefix (e.g., `/api/firewall/` → `"firewall"`). The middleware checks `request.user.permissions[subsystem]`. If the permission level doesn't match the required level, a 403 response is returned.

The `auth` subsystem controls user management. User CRUD endpoints (`/api/auth/users/*`) require `auth: "rw"` ("admin required").

Login-related endpoints are public (no JWT required): `POST /api/auth/login`, `POST /api/auth/refresh`, `POST /api/auth/webauthn/authenticate-begin`, `POST /api/auth/webauthn/authenticate-finish`. (The SPA root `GET /` is also exempt.) Note that for all *other* API routes the middleware requires **both** the `Authorization: Bearer <token>` and `X-Session-Id` headers — a request with only one of the two is rejected with 401.

## Database Layer

Vacuum Wall uses SQLite for authentication and user management data. Subsystem configuration remains as JSON in `config/*/`.

**Architecture:**

- `lib/db.py` — Query ID constants + abstract `Database` baseclass (no SQL strings)
- `lib/db_sqlite.py` — `QUERY_MAP` (query_id → SQLite SQL) + concrete implementation
- Subsystems call by **query ID only** — never write SQL

The abstract `Database` baseclass provides:
- Connection caching via `self.conn` property (lazy initialization)
- Prepared statement auto-cache (cached on first use, reused subsequently)
- `query(query_id, params)` — returns row dicts
- `run(query_id, params)` — returns rowcount
- `run_one(query_id, params)` — returns last_insert_id
- `in_transaction()` context manager — provides `BEGIN`/`COMMIT`/`ROLLBACK` with auto-commit suppressed inside

Environment variables (not config files) control database access:

| Env Var | Default | Description |
|---|---|---|
| `VACUUM_WALL_DB_BACKEND` | `sqlite` | Database backend selection |
| `VACUUM_WALL_DB_PATH` | `data/auth.db` | SQLite database file path |

Flask (`webui/server.py`) calls `get_db()` once at startup to open (and initialize) the database. The daemon does **not** call `get_db()` at startup — it reaches the database lazily through `lib.auth` / `lib.auth_users` the first time an auth operation actually runs. Each process opens its own connection to the same DB file. SQLite WAL mode enables concurrent reads; writes are serialized by SQLite.

## Install-Time Templating

System configuration files in `system/` are Jinja2 templates rendered by `scripts/install.sh` at install time:

- **`systemd/vacuum-wall.service`**, **`systemd/vacuum-walld.service`**, **`systemd/vacuum-wall-acme.service`** — `{{ USER_NAME }}`, `{{ USER_DAEMON_NAME }}`, `{{ USER_GROUP }}`, `{{ PROJECT_DIR }}`, `{{ ACME_HOME }}` are substituted to produce the final systemd unit files installed to `/etc/systemd/system/`. The `PROJECT_DIR` template variable is set from the `INSTALL_DIR` environment variable (defaults to the repo root).
- **`sudoers.d/vacuum-walld`** — `{{ USER_DAEMON_NAME }}` is substituted to produce the sudoers whitelist for the daemon user.
- The timer file (`vacuum-wall-acme.timer`) contains no variable paths and is installed as-is.

Runtime templates (`system/nginx/*.conf`, `system/dnsmasq.conf`, `system/wireguard*.conf`) are rendered at runtime by `lib/` modules via Jinja2 with Python data.

## State Management

Vacuum Wall uses a declarative configuration model. Persistent user-facing configuration lives in `config/<subsystem>/config.json`. Runtime artifacts and generated files live in `data/<subsystem>/`. The application renders these declarations into the format expected by the underlying system service.

| Subsystem | Declarative Config | Runtime Data | Rendered Target | State Persistence |
|---|---|---|---|---|
| firewalld | `config/firewall/config.json` | `data/firewall/rules.json` | N/A (commands issued directly to firewalld via D-Bus) | firewalld manages its own persistent state in `/etc/firewalld/`. `config.json` is the declarative source of truth. `rules.json` is a pre-apply recovery snapshot (`{timestamp, default_zone, zones, config}`) written before every apply; `zones` is the permanent firewalld zone view. |
| dnsmasq | `config/dnsmasq/config.json` | `data/dnsmasq/fragments/` | `/etc/dnsmasq.d/vacuum-wall.conf` | The JSON file is the source of truth. The rendered `.conf` file is overwritten on each apply. |
| nginx | `config/nginx/config.json` | `data/nginx/.htpasswd`, `data/nginx/sites-enabled/` | `data/nginx/sites-enabled/<domain>.conf` + `/etc/nginx/conf.d/vacuum-wall.conf` | All proxy and management domain definitions are derived from the JSON config. Generated `.conf` files are overwritten on each apply. |
| WireGuard | `config/wireguard/config.json` | `data/wireguard/` | `/etc/wireguard/<ifname>.conf` — per-class `wg-<class>.conf` in multi-interface mode; legacy single-interface `wg0.conf` | The JSON file defines the interface, `access_classes`, and all peers. In multi-interface mode each class with assigned peers renders to its own `/etc/wireguard/wg-<class>.conf` (class interface `wg-<class>`) and is brought up independently; apply temps live in `/run/vacuum-wall/`. Rendered configs are overwritten on each apply. |
| networkd | `config/network/config.json` | `data/networkd/` | `/etc/systemd/network/99-<name>.network` | The JSON file defines per-interface static addresses, routes, DNS, DHCP, and link settings. Each entry renders to a `99-<name>.network` INI file. Stale files are cleaned on apply. Public DNS servers are auto-synced to dnsmasq upstreams. |
| ACME | `config/acme/config.json` | `data/acme/` | Certificate and key files | acme.sh manages its own state, renewal scheduling, and account keys. Vacuum Wall triggers issuance and renewal but does not maintain independent ACME state. Account registration (email, CA provider) is stored in the declarative config. |

#### Background Polling

The daemon runs background polling tasks for subsystems with external runtime state. Each subsystem has a configurable interval and a two-layer diff (structural vs volatile) to minimize unnecessary broadcasts.

| Subsystem | Interval | Rationale |
|-----------|----------|-----------|
| firewall | 30s | Most expensive collector (6+ subprocess calls) |
| wireguard | 10s | Peer connections/handshakes change frequently |
| dnsmasq | 10s | Lease file + service status |
| networkd | 10s | Interface up/down, DHCP address changes |
| system | 1s | Real-time metrics (load/memory/swap/traffic) |
| nginx | 60s | Drift re-collection — the collector is a **pure re-read** of the JSON config and rendered artifacts on disk, so polling re-collects manual edits and out-of-band applies. The one-shot nginx legacy-format migration itself is *not* part of the read path — it runs once at startup via `lib/bootstrap.py` (`nginx.migrate_config_file()`). |
| acme | 300s | Drift re-collection — the collector is a **pure re-read** of acme.sh state. The only self-heal in the ACME path is `normalize_acme_home()` (reopens group access on the acme.sh tree, which acme.sh hardens to owner-only on every run). |

All 7 state subsystems are polled. `auth` is **not** a state subsystem at all — auth data lives in the SQLite DB and is fetched on demand by Flask and the daemon, so it has no poll loop and no WS stream.

**Two-layer diff:** Each poll cycle classifies changes as:
- **Structural change** (zones added, peers removed, config changed): triggers `bump()` + broadcast `{"type": "versions", "subsystem": ..., "data": ...}` → daemon pushes the full subsystem data over WS; the client patches the model in place via `modelSet`
- **Volatile change only** (transfer counters, DHCP-assigned IPs): sends `{"type": "tick", "subsystem": ..., "data": ...}` → same in-place patch, without a version bump
- **No change**: silence

Volatile fields per subsystem, defined per collector via `register_volatile()`: `system` (load/memory/swap/traffic), `firewall` (`interfaces[].ips` / `interfaces[].ipv6`), `networkd` (`interfaces[].addresses` only), `wireguard` (peer transfer/handshake stats — both the combined `status.peers[].*` and the per-class `status.classes[].peers[].*`).

Poll intervals are configurable via `VACUUM_WALL_POLL_INTERVALS` env var (`firewall:30,wireguard:10,...`).

On collector failure **during a poll**, no broadcast is sent (avoids noisy ticks) and the existing state is **kept** — `poll()` returns "no change" and does not clear the stored data (only `populate()` clears a subsystem's state to `None` when its collection fails). `broadcast_versions` additionally skips a `None` payload defensively (a null payload would clobber good client data — the next successful poll or mutation broadcasts the real value).

## Apply Bookkeeping and Pending Changes

Every config-backed subsystem records its last-applied state in two keys inside
its declarative JSON: `_last_applied_hash` (SHA-256 of the meta-stripped config)
and `_last_applied_config` (a snapshot of the config at apply time). The helpers
live in `lib/common.py`:

- **`stamp_applied(cfg)`** — writes both keys. Called by each subsystem's apply
  handler after a successful apply.
- **`compute_pending(cfg)`** — returns `(pending, diff)`. Pending when the hash
  is missing or stale; `diff` is a field-level `deep_diff()` between the
  recorded snapshot and the current (meta-stripped) config.
- **`strip_apply_meta(cfg)` / `config_hash(cfg)`** — ignore the bookkeeping keys
  when hashing or comparing configs.
- **`revert_to_applied(path)`** — rewrites a config file from its
  `_last_applied_config` snapshot (re-stamped so the pending check reports it as
  up to date); returns a reason instead when no baseline is recorded (never
  applied).

The `status` handler exposes this cross-subsystem:

- **`GET /status/pending`** — aggregates pending changes per subsystem (the
  firewall section additionally carries the advisory `uncovered_interfaces`
  list).
- **`POST /status/apply-all`** — applies pending subsystems in dependency order
  — **networkd → firewall → wireguard → dnsmasq → nginx** — calling each
  subsystem's apply handler. `{"force": true}` is forwarded only to the
  firewall apply, where it overrides the management-lockout and
  interface-coverage guards.
- **`POST /status/cancel-all`** — reverts every pending subsystem's config file
  to its last-applied snapshot (subsystems with no recorded baseline are
  skipped with a reason). Cancel touches only the declarative config files —
  it never runs live-system commands.

## System Config Import

On daemon startup, `vacuum-walld` runs `import_all()` from `lib/system_import.py`
to reconcile live system configuration with the declarative JSON configs. This
ensures that configurations created by `scripts/install.sh` or edited manually
in system files are imported into the JSON source of truth, preventing drift.

Each subsystem import function parses the corresponding live system config and
updates the JSON config when they differ:

- **`import_dnsmasq`**: Parses `/etc/dnsmasq.d/vacuum-wall.conf` (managed
  block between comment markers) → `config/dnsmasq/config.json`. Only writes
  if config doesn't exist or differs.
- **`import_wireguard`**: Parses `/etc/wireguard/wg0.conf` →
  `config/wireguard/config.json`. Skips if configs match.
- **`import_networkd`**: Globs **all** `/etc/systemd/network/*.network` files →
  `config/network/config.json`, stripping any numeric priority prefix from the
  filename (`99-eth0.network` → `eth0`; `eth0.network` → `eth0`). Only
  adds/updates interfaces; doesn't remove interfaces without a file (they may
  be pending apply).
- **`import_nginx`**: **Skips entirely if `config/nginx/config.json` already
  exists** — it only bootstraps the declarative config from rendered
  `data/nginx/sites-enabled/*.conf` (vacuum-wall-managed files identified by
  the `# Auto-generated by Vacuum Wall` header; `_acme-challenge.conf` is
  skipped) on hosts where the JSON config is absent. When the config exists it
  wins: re-parsing generated server blocks is lossy (backend references get
  flattened to inline paths).
- **`import_firewall`**: Runs `sudo firewall-cmd --list-all-zones` →
  `config/firewall/config.json`. Only writes if no config file exists
  (firewalld state always takes precedence).

All imports are **idempotent** and **non-destructive**: they only write when
configs differ, skip on failure (logged as warnings), and never abort daemon
startup. The returned list of updated subsystems is logged for debugging.

## Cross-Subsystem Sync Event Bus

When a subsystem's configuration changes, related subsystems are automatically
updated to stay consistent. An in-process event bus (`lib/sync.py`) decouples
subsystems — no handler calls into another handler's logic directly.

### How It Works

1. A mutation handler saves its config (e.g., adding a DHCP range).
2. The handler emits a `SyncEvent` on the event bus.
3. Subscribers react by updating related subsystem configs:
   - **DnsToFirewallSync**: Adds `dhcp`, `dns` services to the firewall zone
      for each interface serving a DHCP range. Back-propagates gateway (interface
      IP) into DHCP ranges so clients receive their default route.
    - **WgToFirewallSync**: Manages **per-access-class** firewall zones: for each
       access class with peers, ensures a `vpn-<key>` zone exists with the class's
       WireGuard interface (`wg-<key>`), masquerade enabled, and a UDP accept
       rich rule on the class's `listen_port` (default 51820). Classes with
       `lan_access: true` additionally get inter-zone accept rules for internal
       subnets (derived from zones without masquerade); `lan_access: false`
       (internet-only) classes get no internal rules. Stale class zones
       (`vpn-<key>` whose class no longer has peers) have their WireGuard-created
       entries cleaned up. The single `vpn`/51820 zone is managed **only as a
       legacy fallback** when peers exist without an `access_class`; when
       WireGuard is fully inactive all WireGuard-created entries (interface,
       masquerade, `_source: wg` rules) are removed from the legacy zone.
    - **FirewallToDhcpSync**: **Never deletes** DHCP ranges. Ranges whose
       interface no longer belongs to any zone are kept in config and flagged
       inactive (advisory warning). The only mutation is backfilling the
       `gateway` (interface IP) on ranges for zones with masquerade enabled.
       Zones with the `dhcp` service but no range are logged.
    - **NetworkToAllSync**: Suggests DHCP ranges for static-IP interfaces without
       ranges (advisory only). New network interfaces are logged/flagged but
       **never added** to zones; the only mutation is removing interfaces no
       longer present in the network config from the zones that still list them.
4. The handler refreshes state for the originating subsystem plus all
    transitively affected subsystems.

### Guard Rails

- **Idempotency**: Each subscriber reads current state, computes desired state,
   writes the diff. Running twice is safe.
- **No loops**: The event bus tracks `(subsystem, action)` per dispatch cycle.
   Re-entrant emits for the same key are silently dropped.
- **Firewall-cmd separation**: Sync subscribers only write JSON config. They
   do NOT call `firewall-cmd`. The user clicks "Apply" on the firewall page to
   push to firewalld.
- **Error handling**: Subscriber exceptions are caught, logged as warnings,
   and do NOT abort the originating handler.

### Frontend Impact

Minimal. The sync happens transparently in the backend. The "pending changes"
indicator on the firewall page will show pending when DHCP or WireGuard saves
(since sync writes JSON but does not call firewall-cmd).

## Daemon Startup Order

The daemon's `main()` (`daemon/server.py`) runs a fixed startup sequence after
the aiohttp app is listening on the Unix socket and WebSocket port:

1. **`system_import.import_all()`** — reconciles live system configs into the
   declarative JSON (see System Config Import above). Runs **first** because it
   must see absent config files in order to adopt live system state on first
   start.
2. **`bootstrap()`** (`lib/bootstrap.py`) — creates the runtime `config/` +
   `data/` directories for all subsystems and persists the one-shot nginx
   legacy-format migration (`nginx.migrate_config_file()`). Idempotent. It
   deliberately never creates config *files*: `get_config` reads are pure
   (missing file → in-memory defaults), so files are materialized on the first
   `save_config` (or by the import itself).
3. **`normalize_acme_home()`** — reopens group access on the acme.sh tree
   (acme.sh hardens it to owner-only on every run); a failure here is logged,
   never fatal.
4. **First `state_store.populate()`** — collects all subsystem state; the
   version counter of every successfully populated subsystem is bumped so the
   first WS snapshot is followed by a `versions` broadcast.
5. **`start_polling(loop)`** — spawns one background poll task per subsystem
   (intervals per Background Polling above).

## Directory Structure

### Config — Declarative Settings

Config files are persistent, user-editable JSON that defines the desired state for each subsystem:

```
config/
├── auth/
│   └── config.json                   # JWT settings, WebAuthn RP configuration
├── dnsmasq/
│   └── config.json                   # DHCP ranges, static leases, DNS forwarding, custom records
├── firewall/
│   └── config.json                   # Firewall zones, rich rules, forward ports
├── nginx/
│   └── config.json                   # Proxy domain definitions, management domain, SSL settings
└── wireguard/
    └── config.json                   # WireGuard interface and peer configuration
├── network/
│   └── config.json                   # Per-interface static IP, routes, DNS, DHCP settings
```

### Data — Runtime Artifacts

The `data/` directory holds generated files, credentials, and subsystem artifacts:

```
data/
├── auth.db                          # SQLite database: users, permissions, token_blacklist, refresh_tokens, webauthn_creds, init_sequence
├── nginx/
│   ├── .htpasswd                     # HTTP Basic credentials for basic-authed proxy domains (created on demand; the management UI itself uses JWT only)
│   └── sites-enabled/                # Generated nginx server block .conf files (one per domain)
├── dnsmasq/
│   └── fragments/                    # User-defined dnsmasq config fragments (appended verbatim)
├── firewall/
│   └── rules.json                    # Pre-apply firewall recovery snapshot
├── acme/                             # ACME certificate files (acme.sh home)
├── logs/
│   └── vacuum-wall.log              # Application log file
└── wireguard/                        # WireGuard runtime artifacts
├── networkd/                         # Generated 99-<name>.network files
```

Both `config/` and `data/` reside within the project directory. The systemd service unit's `ReadWritePaths` directive grants the processes write access to these directories, while keeping the rest of the filesystem read-only. The `INSTALL_DIR` value is templated into the service unit at install time.

The daemon uses a **runtime directory** at `/run/vacuum-wall` (created by systemd `RuntimeDirectory=`) for secure temporary files during config apply. `tempfile.NamedTemporaryFile` writes to this directory before `sudo cp` moves files to their final destination, eliminating TOCTOU symlink races that would exist with `/tmp`. The directory is automatically removed on service stop.

`/run` is a fresh tmpfs at every boot, so volatile runtime paths must be recreated at startup. This is a hard requirement, not a best practice: with `ProtectSystem=strict`, namespace setup fails (`226/NAMESPACE`) and the unit crash-loops if any `ReadWritePaths=` entry does not exist when the unit spawns. Each `/run` path the daemon references therefore needs a boot-time creator: the unit's `RuntimeDirectory=vacuum-wall nginx` covers the daemon-owned directories, and the `system/tmpfiles.d/vacuum-wall.conf` spec (installed to `/etc/tmpfiles.d/`) pre-creates `/run/firewalld` and `/run/nginx.pid` at early boot via `systemd-tmpfiles-setup.service` (in practice firewalld creates it itself, and it starts before the daemon; nginx rewrites the pid file on start). `/run/nginx.pid` is additionally listed in the unit's `ReadWritePaths=`: the daemon's `nginx -t` opens the pid file for *writing*, so a read-only mount would fail every daemon-side `nginx -t` (and therefore `/nginx/apply`) with EROFS — the tmpfiles entry guarantees the file exists at spawn. `/run/sudo` is deliberately *not* in the unit's `ReadWritePaths=`: the daemon's sudo children use the NOPASSWD whitelist and never read or write sudo's session directory, so listing it only added a boot-time and restart-time failure mode (sudo removes `/run/sudo` when the last session ends).

## File System Layout

The following file system locations are used for integration with system services:

| Path | Purpose | Managed By |
|---|---|---|
| `/etc/nginx/conf.d/vacuum-wall.conf` | Include directive that pulls in `data/nginx/sites-enabled/*.conf`. | Vacuum Wall (lib/nginx.py) |
| `/etc/nginx/snippets/vacuum-wall-ssl.conf` | Shared SSL configuration snippet (protocols, ciphers, DH parameters, OCSP). Included by all HTTPS server blocks. | Vacuum Wall (lib/nginx.py) |
| `/etc/dnsmasq.d/vacuum-wall.conf` | Generated dnsmasq configuration file. Written from `config/dnsmasq/config.json`. | Vacuum Wall (lib/dnsmasq.py) |
| `/etc/wireguard/<ifname>.conf` | Generated WireGuard interface configuration, written from `config/wireguard/config.json`. Per-class `wg-<class>.conf` in multi-interface mode; legacy single interface `wg0.conf`. | Vacuum Wall (lib/wireguard.py) |
| `/etc/systemd/network/99-<name>.network` | Generated systemd-networkd drop-in files. Written from `config/network/config.json`, one per interface. | Vacuum Wall (lib/network.py) |
| `/etc/sudoers.d/vacuum-walld` | Sudo whitelist for the daemon user. Defines all permitted privilege escalations. | Install script (rendered from Jinja2 template) |
| `/run/vacuum-wall` | Runtime directory for secure temp files during config apply (nginx, dnsmasq, wireguard, networkd). Created by systemd `RuntimeDirectory=`, removed on stop. | Daemon (systemd unit) |
| `/run/nginx` | Runtime directory referenced by the daemon's `ReadWritePaths=`; must exist at spawn. Created by systemd `RuntimeDirectory=` before namespace setup. | Daemon (systemd unit) |
| `/run/nginx.pid` | nginx pid file. Must exist at spawn **and** be writable by the daemon: its `nginx -t` opens the file for writing, so it needs both a boot-time creator (`system/tmpfiles.d/vacuum-wall.conf`) and a `ReadWritePaths=` entry (nginx rewrites it on start). | nginx / systemd-tmpfiles (early boot) |
| `/run/firewalld` | Root-owned runtime dir of firewalld. Must exist at spawn because of `ProtectSystem=strict` + `ReadWritePaths=` (see volatile-/run note above). Present while firewalld runs; also pre-created at early boot by `system/tmpfiles.d/vacuum-wall.conf`. | firewalld / systemd-tmpfiles (early boot) |
| `/run/sudo` | sudo's session directory. Present only while sudo sessions exist. **Not** in the unit's `ReadWritePaths=` (NOPASSWD sudo children never need it) — see volatile-/run note above. | sudo (created/removed on demand) |
| `data/auth.db` | SQLite database: users, permissions, token_blacklist, refresh_tokens, webauthn_creds, init_sequence. Created on first access via `get_db()`. | Auth layer (lib/db.py) |

The `/etc/nginx/conf.d/vacuum-wall.conf` include file ensures that all domain-specific configurations in `sites-enabled/` are loaded by nginx without modifying the main `nginx.conf`. The SSL snippet keeps TLS settings consistent across all managed domains and allows global updates from a single location.

## Frontend Architecture

The web UI is a single-page application built on **Hoover**, a custom lightweight VDOM framework. See [Hoover Framework Reference](hoover.md) for the complete API.

### Request Flow (Frontend)

```
Client requests / ──→ nginx ──→ Flask (serves index.html)
Client loads /static/app.js ──→ served by nginx directly from disk (mgmt `location /static/` alias, no Flask round-trip) ──→ Hoover initializes, auth model 'check' fetch action (GET /api/auth/session; not-ok with a stored refresh token → exactly one refresh) → if no valid session, render #login
Authenticated ──→ mounts #sidebar and #main render roots
apiFetch() ──→ injects BOTH Authorization: Bearer <token> and X-Session-Id headers ──→ Flask REST API
Flask before_request ──→ validates JWT + session ID from headers, checks blacklist, verifies permissions
Hoover connects WebSocket ──→ daemon/ws (raw JWT as Sec-WebSocket-Protocol subprotocol name; legacy `Bearer <token>` subprotocol + X-Auth-Token header fallbacks accepted)
Page navigate (hash change) ──→ reactive router state updates ──→ render engine re-executes ──→ VDOM diff patches DOM
User action (form submit) ──→ apiFetch() ──→ Flask REST API ──→ daemon/client.py ──→ vacuum-walld
Token expiry ──→ TTL-driven scheduleRefresh() (timer at access-token TTL − 60s, min 30s) ──→ POST /api/auth/refresh ──→ new tokens
WS connect ──→ snapshot (full state) / versions + tick deltas (per-subsystem data) ──→ modelSet() patches model in place ──→ render engine VDOM-diffs and patches only changed DOM nodes
WS close ×3 ──→ refreshAuth() + reconnect; 2 consecutive failed refresh+reconnect episodes ──→ give up: no more reconnect attempts until the page is reloaded (REST API keeps working)
```

The SPA entry point only serves `index.html` at `/`. All other paths return 404. Non-API, non-static paths are not served by Flask — the client-side router handles all navigation via hash changes. A dedicated `/vendor/<path>` route serves vendored JS libraries. On the management domain, nginx serves `/static/` directly from `webui/static/` via a `location /static/` alias in the generated server block, so asset requests never reach Flask in production; Flask's static route remains as the dev-mode fallback.

### Component Model

Each route is a `definePage()` component with reactive state, async data loading, and WebSocket auto-refresh. Pages are mounted using `hComp(page, key)` in the router, where the key determines lifecycle boundaries. The same key reuses the component instance (preserving state); a different key unmounts the old page and mounts the new one.

### No Build Step

All JavaScript is served as ES modules. Cache invalidation is handled via HTTP cache-control headers: the management domain's `/static/` assets carry `Cache-Control: no-cache` (browsers revalidate every load; unchanged files return 304 via nginx's built-in ETag), so updates are picked up on the next page load. Dev mode (`VACUUM_WALL_DEV`) uses short TTLs instead.

Because there is no build step, backend changes can be hot-reloaded too: the `vacuum-wall` systemd unit defines `ExecReload=` which sends SIGHUP to the Flask process — Flask auto-reloads its `webui.*` and `lib.*` modules and then restarts itself, so `systemctl reload vacuum-wall` picks up code changes without a full stop/start.

### WebSocket Data Streaming

The daemon pushes state over the WebSocket — no HTTP round-trip for auto-refresh. On connect, after the JWT handshake, it sends a full snapshot (`{"type": "snapshot", "data": {subsystem: state|null, …}}`). On every structural change it broadcasts a per-subsystem delta (`{"type": "versions", "subsystem": …, "data": …}`); on volatile-only changes it sends `{"type": "tick", "subsystem": …, "data": …}`. The client's `handleMessage` patches the matching reactive model in place via `modelSet()`, and the VDOM diff touches only the changed nodes. HTTP remains the fallback for the initial load (3s timer) and for reconnect recovery.

## Firewall Interface-Coverage Invariant

A core apply-path guarantee: every network-managed interface (`lo` and `wg*` excluded) must be covered by a zone in `config/firewall/config.json` **or** listed under the top-level `unmanaged` key. The config is the source of truth for zone interfaces — an omitted `interfaces` key counts as an empty list, so there are no hands-off zones.

The check is the pure `lib.firewall.validate_coverage()`, enforced at:

- **Save time** — `POST`/`PATCH /firewall/config` returns 400 when the proposed config would leave an interface uncovered.
- **Apply time** — `POST /firewall/config/apply` returns 409 on coverage failure; `{"force": true}` (e.g. from the status apply-all endpoint) overrides the guard.
- **Live drift is advisory only** — the firewall state carries an `uncovered_interfaces` field and the status pending summary surfaces it, but live coverage never blocks a save or apply on its own.

## Zone Model

The firewalld zone layout in Vacuum Wall follows a defense-in-depth approach, segmenting traffic based on trust level:

| Zone | Interfaces | Trust Level | Description |
|---|---|---|---|
| `public` / `external` | WAN (e.g., `eth0`) | Untrusted | Internet-facing. Only explicitly allowed inbound services (HTTPS/443, WireGuard/51820, ICMP echo rate-limited) are accessible. All other inbound traffic is dropped. |
| `internal` | LAN (e.g., `eth1`) | Trusted | Local area network. DHCP (UDP 67/68) and DNS (UDP/TCP 53) are served. Masquerade (NAT) is enabled for outbound Internet access from LAN clients. Inbound from WAN to this zone is not directly accessible. |
| `vpn` | WireGuard (`wg0`) | Semi-trusted | WireGuard tunnel interface. Firewall rules determine which internal services and subnets VPN peers can reach. By default, VPN peers can access the Internet but may be restricted from accessing management interfaces or sensitive LAN services. |
| `trusted` | Management interface | Administrative | Used for management traffic. The `loopback` zone covers localhost communication, enabling the Flask WebUI to receive proxied requests from nginx on `127.0.0.1:9090`. |

### Custom Zones

Additional zones can be created for specialized network segments:

- **DMZ zone**: For hosting public-facing services that need to be isolated from the internal LAN. Traffic from the DMZ to the `internal` zone is denied by default.
- **Guest zone**: For visitor Wi-Fi or untrusted devices. Access is limited to outbound Internet traffic only, with no access to `internal` or `vpn` zones.
- **IoT zone**: For devices requiring restricted outbound access (e.g., blocking telemetry domains).

Each custom zone can define its own source rules, port forwardings, and inter-zone traffic policies. The Flask WebUI provides interfaces to create, modify, and assign interfaces to zones at runtime.