refactor: daemon collectors, thin webui proxies, pure config reads

- move state collectors from lib/state.py to daemon/collectors/ (7
  modules, registration side-effect; daemon/server.py imports the
  package before the first populate())
- webui/api: new daemon_route() decorator factory in common.py
  collapses the try/except daemon-proxy boilerplate in all 8
  blueprints (rules/params/body/transform keep responses identical)
- firewall: interface-coverage invariant — config is the source of
  truth for zone interfaces (absent key = empty, no hands-off
  zones); pure validate_coverage() enforced at save (400) and apply
  (409, force: true overrides), top-level `unmanaged` exemption
- lib: get_config() reads are now pure (no dir creation or writes);
  new lib/bootstrap.py creates runtime dirs and persists the
  one-shot nginx legacy migration at daemon start, after
  system_import (lib.nginx.migrate_config_file)
- lib/common: compute_pending() apply-bookkeeping helper
- daemon: emit_and_refresh() handler helper; refresh_state(bump=) so
  /status/refresh no longer bumps versions (poll/mutation only)
- acme: move --log last so acme.sh never treats a real arg as the
  log-file argument
- docs: AGENTS.md, config.md, state-model.md, api.md updated;
  HARDEN.md dropped (plan implemented); apply-confirm force wording

Tests: 917 passed; ruff check + format clean.
This commit is contained in:
2026-09-03 00:40:56 +00:00
parent 89b64960f3
commit faa076370d
49 changed files with 2834 additions and 3821 deletions
+24 -3
View File
@@ -39,10 +39,15 @@ Basic auth (`.htpasswd`) renders **only** for proxy domains whose
- `daemon/server.py` — aiohttp server, route registry, batch routing, WebSocket broadcast, state refresh, periodic polling.
- `daemon/client.py` — Sync HTTP client over Unix socket using `requests_unixsocket.Session`.
- `daemon/iface.py`**Single source of truth** for all daemon API endpoints. Every endpoint is a frozen `(method, path)` tuple. Renaming here auto-updates both server registry and client calls.
- `daemon/handlers/*.py` — Privileged operation handlers. All `sudo` calls live here.
- `daemon/handlers/*.py` — Privileged operation handlers. All mutating `sudo` calls live here.
- `daemon/collectors/` — Per-subsystem state collectors (7 modules: firewall, dnsmasq, nginx, acme, wireguard, networkd, system). Read-only `sudo` queries that populate `lib.state`. Imported for their registration side-effect; `daemon/server.py` imports the package before the first `populate()`.
- `lib/state.py` — In-memory state store with per-subsystem collectors. Populated at daemon startup, refreshed on mutation/poll. Backs the WebSocket push stream: `get_snapshot()` (full state on WS connect), `poll()` two-layer diff (structural `versions` broadcast vs volatile-only `tick` broadcast, per-subsystem, each carrying the full subsystem data), `register_volatile(subsystem, keys)` to mark volatile fields, `get_versions()`/`bump()`. Per-subsystem poll intervals via `_DEFAULT_POLL_INTERVALS` (system 1s, firewall 30s, wireguard/dnsmasq/networkd 10s, nginx 60s, acme 300s).
- `lib/common.py` — Shared utilities: `run()`, `run_proc()`, `load_json()`, `save_json()`, `deep_merge()`, `ensure_dirs()`, `config_hash()`, `validate_interface_name()`.
- `lib/common.py` — Shared utilities: `run()`, `run_proc()`, `load_json()`, `save_json()`, `deep_merge()`, `ensure_dirs()`, `config_hash()`, `validate_interface_name()`, plus the apply-bookkeeping helpers `stamp_applied()` / `strip_apply_meta()` / `compute_pending()` / `deep_diff()` / `revert_to_applied()` (the `_last_applied_hash` / `_last_applied_config` keys every config-backed subsystem uses for pending-change detection and cancel-all).
- `lib/logging.py` — Logging setup used by both webui and daemon. Reads `VACUUM_WALL_LOG_LEVEL`.
- `lib/sync.py` — Cross-subsystem sync event bus (in-process pub/sub); handlers emit events on mutation and subscribers refresh affected subsystems.
- `lib/system_import.py` — Startup system-config import/reconcile: parses native config sources and merges them into the declarative JSON on daemon start.
- `lib/bootstrap.py` — Daemon-startup filesystem bootstrap, run **after** `system_import.import_all()` (which must see absent config files to adopt live state on first start) and before the first state collection: creates the runtime `config/`+`data/` directories and persists the one-shot nginx legacy-format migration. Never creates config files (reads stay pure; files appear on first `save_config`).
- `lib/schema.py` — TypedDict state schemas for the per-subsystem state payloads.
- `lib/*.py` — Backend modules (parsing, config, shared logic). Full type hints and `__all__` exports. No sudo calls.
- `vendor/` — Vendored scripts and JS libraries (`acme.sh`, `htm`).
- `data/` — Runtime artifacts (generated .confs, `.htpasswd`, ACME certs, firewall backup, dnsmasq fragments).
@@ -118,6 +123,21 @@ Reload running Flask via SIGHUP (auto-reloads `webui.*` and `lib.*` modules, the
Pattern for mutations: write JSON → render native config → `sudo <cmd>` to apply.
Adding a new privileged command requires a sudoers entry **and** the `daemon/handlers/` code.
**Config reads are pure.** Every `lib/<subsystem>.get_config()` is a side-effect-free
read (returns in-memory defaults when the file is missing; nginx applies its
legacy-format migration in memory). State collectors therefore never write to
disk — filesystem setup (runtime dirs, one-shot nginx migration) happens once at
daemon startup in `lib/bootstrap.py` (after `system_import.import_all()`).
**Firewall interface-coverage invariant.** Every network-managed interface
(`lo`/`wg*` excluded) must be covered by a zone in `config/firewall/config.json`
or listed under the top-level `unmanaged` key. The config is the source of truth
for zone interfaces (an omitted `interfaces` key = empty list; no hands-off
zones). Enforced at save time (`POST`/`PATCH /firewall/config` → 400) and apply
time (`POST /firewall/config/apply` → 409, `force: true` overrides) via the pure
`lib.firewall.validate_coverage()`. Live drift is advisory only
(`uncovered_interfaces` state field). See `docs/config.md`.
## API Response Contract
- Success: `{"ok": true, "data": <value>}``_ok(data)` (Flask) or `ok(data)` (aiohttp)
@@ -140,7 +160,7 @@ user (full `rw` on all subsystems; default username `admin`), **not** an nginx h
**Linter / formatter:** Ruff (`ruff check` + `ruff format`). Config in `pyproject.toml`.
**Tests:** pytest in `tests/` (17 modules). All subprocess calls are mocked — no system services required.
**Tests:** pytest in `tests/` (27 test files). All subprocess calls are mocked — no system services required.
```bash
.venv/bin/ruff check lib/ webui/ tests/ # lint
@@ -160,4 +180,5 @@ user (full `rw` on all subsystems; default username `admin`), **not** an nginx h
| `docs/config.md` | JSON schema for each subsystem config (dnsmasq, nginx, wireguard, cert types) |
| `docs/api.md` | REST API endpoint reference, request/response contracts, route patterns |
| `docs/hoover.md` | Custom frontend framework API reference |
| `docs/state-model.md` | Per-subsystem state schema, the `versions`/`tick` two-layer diff, and the pending-changes model |
| `docs/overview.md` | Subsystem summaries, tech stack, complete project directory tree |