ws: migrate push stream to data streaming

- daemon: send full snapshot on connect; versions/tick now carry the
  full state of one subsystem (subsystem + data); no legacy
  updated/subsystems payloads; refresh_state and POST /status/refresh
  broadcast per-subsystem versions with data
- client: modelSet() patches models in place; onMessage/topic refresh
  retired; 3s initial-load fallback via new POST /api/status/refresh
- schema: lib/schema.py TypedDicts + hoover/schema.js defaults +
  docs/state-model.md as single source of truth for state shapes
- system: poll at 1s, volatile metrics registered, dashboard uses a
  dedicated system model (status model removed)
- firewall: refuse to strip both https and ssh from the default zone
  (409, force override via UI confirm); set_zone_services persists
  services to the declarative config; collector exposes default_zone
- UI: pages migrate to flat state shapes; post-mutation modelFetch
  refreshes removed (WS delta covers it)
- tests: ws snapshot/delta/broadcast, refresh-state, schema types,
  model-set/js ws handler and reconnect fallback
This commit is contained in:
2026-08-20 01:38:00 +00:00
parent 9c9f92ad04
commit 332d14e37d
45 changed files with 2819 additions and 496 deletions
+15 -4
View File
@@ -8,7 +8,7 @@ Deploys on Debian 13 (trixie). Serves from repo root by default.
## Architecture
```
Client ──→ nginx (SSL + basic auth) ──→ Flask (127.0.0.1:9090)
Client ──→ nginx (TLS; basic auth on basic-authed proxy domains only) ──→ Flask (127.0.0.1:9090)
Flask ──→ daemon/client.py (Unix socket) ──→ vacuum-walld (aiohttp, daemon.sock)
vacuum-walld ──→ daemon/handlers/*.py ──→ sudo <cmd> ──→ system service
```
@@ -16,6 +16,15 @@ vacuum-walld ──→ daemon/handlers/*.py ──→ sudo <cmd> ──→ syste
Blueprints are thin proxies — they never call `lib/` directly. All operations flow
through the daemon client over a Unix socket.
**Authentication:** the management UI gets **no** nginx-level `auth_basic` — the mgmt
server block's `location /` is a bare proxy and `/ws` is `auth_basic off`. Management
auth is the Flask-layer JWT middleware (`POST /api/auth/login`
`Authorization: Bearer <token>`; public paths: static files, `/vendor/`, auth endpoints)
plus the daemon WS handshake (raw JWT as the `Sec-WebSocket-Protocol` subprotocol).
Basic auth (`.htpasswd`) renders **only** for proxy domains whose
`config/nginx/config.json` has an `auth` block — never for the management domain
(see `docs/security.md`, "Management Interface").
### Two-User Model with Shared Group
- **`vacuum-walld`** (daemon user): runs the privileged background daemon with `NOPASSWD sudo` whitelist (`/etc/sudoers.d/vacuum-walld`). Owns socket. Primary group is the WebUI user's primary group. Daemon user name is derived: `USER_NAME` + `d`.
@@ -31,7 +40,7 @@ through the daemon client over a Unix socket.
- `daemon/client.py` — Sync HTTP client over Unix socket using `requests_unixsocket.Session`.
- `daemon/iface.py`**Single source of truth** for all daemon API endpoints. Every endpoint is a frozen `(method, path)` tuple. Renaming here auto-updates both server registry and client calls.
- `daemon/handlers/*.py` — Privileged operation handlers. All `sudo` calls live here.
- `lib/state.py` — In-memory state store with per-subsystem collectors. Populated at daemon startup, refreshed on request. Backs WebSocket versioning/broadcast.
- `lib/state.py` — In-memory state store with per-subsystem collectors. Populated at daemon startup, refreshed on mutation/poll. Backs the WebSocket push stream: `get_snapshot()` (full state on WS connect), `poll()` two-layer diff (structural `versions` broadcast vs volatile-only `tick` broadcast, per-subsystem, each carrying the full subsystem data), `register_volatile(subsystem, keys)` to mark volatile fields, `get_versions()`/`bump()`. Per-subsystem poll intervals via `_DEFAULT_POLL_INTERVALS` (system 1s, firewall 30s, wireguard/dnsmasq/networkd 10s, nginx 60s, acme 300s).
- `lib/common.py` — Shared utilities: `run()`, `run_proc()`, `load_json()`, `save_json()`, `deep_merge()`, `ensure_dirs()`, `config_hash()`, `validate_interface_name()`.
- `lib/logging.py` — Logging setup used by both webui and daemon. Reads `VACUUM_WALL_LOG_LEVEL`.
- `lib/*.py` — Backend modules (parsing, config, shared logic). Full type hints and `__all__` exports. No sudo calls.
@@ -59,7 +68,7 @@ Conventions:
### Daemon Endpoints
- Unix socket at `data/daemon.sock` (configurable via `VACUUM_WALLD_SOCKET`)
- WebSocket at `127.0.0.1:9091` (configurable via `VACUUM_WALLD_WS_PORT`) for real-time state notifications
- WebSocket at `127.0.0.1:9091` (configurable via `VACUUM_WALLD_WS_PORT`) for real-time state streaming: full `snapshot` on connect, then per-subsystem data-carrying `versions` (structural) / `tick` (volatile-only) deltas. The client patches reactive models in place via `modelSet()` — no HTTP round-trip for auto-refresh.
- Periodic polling per subsystem via `lib.state._DEFAULT_POLL_INTERVALS`, overridable with `VACUUM_WALL_POLL_INTERVALS` env var (format `subsystem:seconds,subsystem:seconds`)
- Start as `python -m daemon.server` or via the `vacuum-walld` console script
@@ -119,7 +128,9 @@ Adding a new privileged command requires a sudoers entry **and** the `daemon/han
## Deploy
`scripts/install.sh` is the single deploy script. Run as root. CLI flags take precedence over env vars.
`MGMT_PASS` is strictly required; `MGMT_DOMAIN` auto-detected from hostname.
`MGMT_PASS` is strictly required — it is the **SQLite DB** password for the initial `admin`
user (full `rw` on all subsystems; default username `admin`), **not** an nginx htpasswd.
`MGMT_DOMAIN` auto-detected from hostname.
### Service Start Order