Compare commits

...

83 Commits

Author SHA1 Message Date
mteehan b503a6dcf0 docs: full refresh per DOCSPLAN (auth subsystem, backends model, access classes, sudo table, state-model mechanics) + 3 stale docstrings 2026-09-05 16:34:57 +00:00
mteehan 78fcb01877 fix: install.sh loop abort, ACME poll sudo gate, /static/ sub-paths
- install.sh: the traversal-chmod loop assigned _d but looped over the
  never-set $d; under set -u every fresh install aborted with
  "d: unbound variable" at that line. Loop over $_d.
- acme collector: the self-heal normalize (sudo chmod g+rwX) now runs
  only when a no-sudo group-read-bit probe detects a lost bit — acme.sh
  re-hardens the tree 600 on every run, so the steady-state poll makes
  no sudo call. The group bit (not daemon readability) is what the
  two-user model keeps for the WebUI user.
- lib.acme: new get_acme_home() accessor (ACME_HOME env, default
  data/acme), reused by _run_acme; _summarize_acme_output preserves a
  "Permission denied" line even when it is not among the final two, so
  the collector's actionable-error matcher keeps firing.
- nginx template: emit location /static/ for any is_management path
  (not only '/'); the SPA references /static/... at the domain root
  regardless of the management backend path.
- tests: probe, summarizer, and nginx-subpath cases in
  test_state.py, test_acme.py, test_nginx.py.
2026-09-05 00:38:34 +00:00
mteehan 6229c39347 ui: declarative per-page tab titles via definePage title 2026-09-04 21:59:09 +00:00
mteehan 6476695d29 fix: ACME cert list self-heals when account.conf is left owner-only
The startup normalize and _run_acme_preflight covered daemon startup and issue/renew, but the recurring collector poll called lib.acme.list_certs() without normalizing ACME_HOME. A non-daemon run (e.g. a manual run as the WebUI user) re-creating account.conf owner-only made every acme.sh --list exit 2, so the collector returned certs=[] and the UI showed no certs until the next issue/renew or daemon restart.

- collector: normalize_acme_home() before list_certs() so the poll self-heals
- issue pre-check: normalize before the direct lib.acme.list_certs()
- _parse_account_conf: read acme.sh v3 account.conf (not just .account.conf)
- _collect_acme: actionable status.error for the account.conf perm case
- install.sh: chown ACME_HOME conf files to the daemon user
2026-09-04 21:38:55 +00:00
mteehan 2b7fe1f485 ui: per-container #comp lifecycle, exp-claim auth refresh TTL
- hoover: #comp registry + expanded-content cache now per render
  container; committing one root no longer unmounts/remounts
  components owned by another root (infinite load loop on pages
  whose load() re-mutates reactive state)
- auth_model: refresh timer scheduled from the token's remaining
  exp claim (unverified decode, mirrors lib/auth.py); falls back to
  the configured TTL for non-JWT/malformed/already-expired tokens
- docs: hoover.md documents both behaviors
- tests: exp-claim TTL cases in test-auth-model.js; new
  test-render-lifecycle.js regression suite
2026-09-03 17:25:22 +00:00
mteehan fc478a016e nginx: serve mgmt /static/ assets from disk (no Flask round-trip)
Generated management-domain server blocks now include a
location /static/ aliasing webui/static/ with Cache-Control: no-cache
(ETag revalidation -> 304), nosniff, and a restrictive CSP, so SPA
asset requests no longer reach Flask. Flask's static route remains
the dev-mode fallback.

- lib/nginx.py + daemon/handlers/nginx.py: static_root render context
  (the handler renders the template directly, so both paths need it)
- template: alias uses a trailing slash (nginx concatenates the
  location remainder onto the alias value)
- install.sh: a+rX on webui/static plus execute-up-the-parent-chain
  so the nginx worker (www-data) can traverse repo-in-$HOME installs
- tests: mgmt static location assertions (positive + non-mgmt negative)
- docs: AGENTS.md, architecture.md, security.md static-serving notes
2026-09-03 14:58:57 +00:00
mteehan d78b90db00 ui: toast per-type durations, Details modal for long errors, concise acme.sh failure summary 2026-09-03 03:29:50 +00:00
mteehan 7e6fd71bdc test: acme --log flag asserts explicit log file path 2026-09-03 03:29:38 +00:00
mteehan faa076370d refactor: daemon collectors, thin webui proxies, pure config reads
- move state collectors from lib/state.py to daemon/collectors/ (7
  modules, registration side-effect; daemon/server.py imports the
  package before the first populate())
- webui/api: new daemon_route() decorator factory in common.py
  collapses the try/except daemon-proxy boilerplate in all 8
  blueprints (rules/params/body/transform keep responses identical)
- firewall: interface-coverage invariant — config is the source of
  truth for zone interfaces (absent key = empty, no hands-off
  zones); pure validate_coverage() enforced at save (400) and apply
  (409, force: true overrides), top-level `unmanaged` exemption
- lib: get_config() reads are now pure (no dir creation or writes);
  new lib/bootstrap.py creates runtime dirs and persists the
  one-shot nginx legacy migration at daemon start, after
  system_import (lib.nginx.migrate_config_file)
- lib/common: compute_pending() apply-bookkeeping helper
- daemon: emit_and_refresh() handler helper; refresh_state(bump=) so
  /status/refresh no longer bumps versions (poll/mutation only)
- acme: move --log last so acme.sh never treats a real arg as the
  log-file argument
- docs: AGENTS.md, config.md, state-model.md, api.md updated;
  HARDEN.md dropped (plan implemented); apply-confirm force wording

Tests: 917 passed; ruff check + format clean.
2026-09-03 00:40:56 +00:00
mteehan 89b64960f3 ui: amber pending-edit markers for unapplied config changes
Add hoover/dirty.js: line-matching helpers that flag UI rows/cards
edited (saved to config) but not yet applied, consuming the pending
state the daemon already streams — status.pending_diff for hash
subsystems, firewall pending zone+type for firewalld. Visual language
is amber (.config-dirty + PendingDot), distinct from the red
.pending-delete; orphanInfo surfaces removed entries (e.g. WireGuard
peers) on their container table. Wired into the backends, dhcp,
interfaces, nat, proxy, rules, wireguard, and zones pages; Card and
Table gain cls/title props. Covered by 27 node tests
(tests/test-dirty.js).
2026-09-01 20:17:15 +00:00
mteehan 75b86fd60d fix: ACME ownership self-heal + daily timer, apply-all force, firewall baseline re-stamp
acme:
- acme.sh chmods its tree to owner-only (700/600) every run, which
  broke the two-user model: a tree left owner-only by one user made
  every acme.sh call of the other exit 2
- normalize_acme_home() reopens group access (sudo chmod g+rwX,
  files only — setgid dirs trip RestrictSUIDSGID); _run_acme_preflight
  is the choke point before every daemon acme.sh call + startup
- acme service now runs as the daemon user; --log persists the raw CA
  transcript; SYS_LOG=6 journals manual issue/renew runs
- timer daily-only: two runs/day landed inside ZeroSSL's 24h
  validation backoff (Retry-After: 86400) — a permanent renewal lockout
- _collect_acme no longer raises on cert-list failure; reports
  status.error (AcmeState.status) so the certs page can surface it

firewall: re-stamp the applied baseline on live zone mutations
(interfaces/services/rich-rules/masquerade/forward-ports) so cancel-all
reverts to post-mutation state, not a stale install-era snapshot;
set_masquerade syncs the declarative config for existing zones;
add_forward_port records toaddr only with toport

status: apply-all accepts {"force": true} (forwarded to the firewall
apply only); ApplyConfirm force checkbox; applyResultToasts() — the
errors map wins over the 200; ActionButton checks errors before the
success toast; dashboard uses ApplyConfirm

system_import: drift re-imports carry the existing apply-meta; first
import stamps the adopted content as applied (it is the running state)
— no phantom pending changes

nginx: get_config only re-saves when migration actually changed the
config (no more owner/mtime churn on every read)

install: repair mis-owned top-level system dirs (tmpfiles
unsafe-path-transition), warn with a full-repair command for deeper
mis-ownership

daemon/server: loop.get_exception_handler() (aiohttp API fix)

tests: 888 pytest + 24 node passing; ruff clean
2026-09-01 02:35:04 +00:00
mteehan ac52918df5 firewall: interface-coverage apply guard, target drift, non-destructive DHCP sync
Post-DHCP-incident hardening per HARDEN.md.

- apply guard: refuse (ConflictError, `force` overrides) when a
  network-managed interface would end up in no zone; absent
  `interfaces` key = hands-off, explicit `[]` = unassign-all
- surface `uncovered_interfaces` in firewall state (lo/wg* filtered)
  + advisory in /api/status/pending; zones.js banner + interfaces-picker
  last-zone confirm
- target drift (Option A): absent or default-normalizing target is
  unmanaged: not diffed, never re-set by apply; create_zone runs
  --new-zone first and sets non-default targets only; importer omits
  the target key for default zones
- FirewallToDhcpSync keeps stale DHCP ranges and flags them instead of
  deleting; `dnsmasq` affected only on a real gateway mutation
- real pre-apply recovery snapshot in data/firewall/rules.json
  ({timestamp, default_zone, zones, config}); drop the empty post-apply
  skeleton
- daemon shutdown: bounded grace for in-flight tasks + suppressed
  teardown exception noise on SIGTERM
- also carries the firewall service-descriptions feature
  (get_service_descriptions + service_descriptions state field + UI)
- tests + docs across firewall/status/state/sync/schema; ruff clean,
  867 passing
2026-08-28 23:38:21 +00:00
mteehan 55309cfd86 status: cancel-all reverts pending changes to last applied config
- lib.common.revert_to_applied(): restore a config file from its
  _last_applied_config snapshot (stamped hash); no baseline -> skip with
  reason, file untouched
- firewall config_apply now stamps the applied baseline like the other
  subsystems; GET /firewall/config and the state collector strip the
  internal _last_applied_* keys
- POST /status/cancel-all + /api/status/cancel-all: revert pending
  subsystems, {cancelled, skipped, errors}, partial-failure safe
- dashboard: "Cancel All Changes" button with confirm modal
  (CancelConfirm, reuses the pending-changes modal rows); the pending
  changes card is hidden entirely when nothing is pending
- tests: revert_to_applied, status_cancel_all, firewall stamping/meta
  stripping, /api/status/cancel-all route, node tests for CancelConfirm;
  firewall _config_apply tests no longer write the real repo config
- docs: api.md, state-model.md, config.md, hoover.md
2026-08-21 02:10:05 +00:00
mteehan 30b51ad7d3 state: applied-config snapshots + per-field pending diffs
- lib/common: stamp_applied() now records a _last_applied_config
  snapshot alongside the hash; strip_apply_meta() centralizes
  bookkeeping-key stripping; deep_diff() reports field-level changes
- state collectors (dnsmasq/nginx/wireguard/networkd) expose
  pending_diff so the dashboard can show exactly which fields
  changed since the last apply (wireguard diff excludes
  private_key paths)
- dashboard pending-changes card renders per-change lines with a
  generic fallback when no snapshot is recorded
- firewall: firewalld built-in zones no longer flagged as
  unmanaged; public-zone masquerade skipped in pending changes
  since apply drives it via nftables propagation
- schema: PendingChange TypedDict; pending_diff on DnsmasqStatus /
  WgStatus; tests in test_common.py, test_firewall.py, test_state.py
2026-08-21 00:59:19 +00:00
mteehan a77cee821b Restart script re-execs itself with sudo when not run as root
The script restarts system services via systemctl, which requires root.
Previously it would only fail per-command when run by an unprivileged
user. Add an EUID guard at the top that re-execs the script through
sudo, preserving the original path and arguments.
2026-08-20 23:05:54 +00:00
mteehan 94705490b9 certs: async renewal via background task + status polling
- POST /acme/renew returns a request_id and spawns a background task
  (renew/deploy/refresh steps); dedups per-domain like issue
- completes as "skipped" when acme.sh reports the renewal window
  has not been reached (no --force)
- new GET /acme/renew/status endpoint (iface + handler + blueprint)
- run acme.sh subprocesses off the event loop (asyncio.to_thread)
  in both issuance and renewal
- ActionCell: busy/busyLabel props; certs page disables the Renew
  button and polls renewal status with toasts for success/skip/fail
2026-08-20 13:10:16 +00:00
mteehan 332d14e37d ws: migrate push stream to data streaming
- daemon: send full snapshot on connect; versions/tick now carry the
  full state of one subsystem (subsystem + data); no legacy
  updated/subsystems payloads; refresh_state and POST /status/refresh
  broadcast per-subsystem versions with data
- client: modelSet() patches models in place; onMessage/topic refresh
  retired; 3s initial-load fallback via new POST /api/status/refresh
- schema: lib/schema.py TypedDicts + hoover/schema.js defaults +
  docs/state-model.md as single source of truth for state shapes
- system: poll at 1s, volatile metrics registered, dashboard uses a
  dedicated system model (status model removed)
- firewall: refuse to strip both https and ssh from the default zone
  (409, force override via UI confirm); set_zone_services persists
  services to the declarative config; collector exposes default_zone
- UI: pages migrate to flat state shapes; post-mutation modelFetch
  refreshes removed (WS delta covers it)
- tests: ws snapshot/delta/broadcast, refresh-state, schema types,
  model-set/js ws handler and reconnect fallback
2026-08-20 01:38:00 +00:00
mteehan 9c9f92ad04 fix: daemon /run spawn hardening, auth guard before first paint, WS refresh cap, interfaces runtime state
systemd: pre-create volatile /run paths so vacuum-walld's ProtectSystem=strict namespace setup cannot fail with 226/NAMESPACE — RuntimeDirectory=vacuum-wall nginx plus a tmpfiles.d spec (installed to /etc/tmpfiles.d/) covering /run/firewalld and /run/nginx.pid. Drop /run/sudo from ReadWritePaths: NOPASSWD children never need it, and its absence crash-looped restarts after sudo removed /run/sudo.

webui: run the auth session check before mounting the shell so logged-out visitors never flash the sidebar or a protected page; router guard and sidebar now react to auth state, and the login page renders full-bleed.

ws: cap refresh->reconnect episodes at 2 consecutive failures; if the WS path stays dead after a token refresh, abandon reconnection instead of looping refreshAuth forever (UI keeps working via REST until reload).

api: GET /api/network/interfaces now includes loopback and returns per-interface {config, runtime}; dashboard reads runtime.state (carrier counts as up) and the interfaces page filters lo client-side.

daemon: re-collect nginx state after lazy config migration (cached list went stale when the on-disk format changed under it), skip system_import.nginx when config.json already exists (re-parsing vacuum-wall's own generated sites is lossy), and poll nginx (60s) / acme (300s) state so file drift self-heals.
2026-08-19 15:32:36 +00:00
mteehan 4bd4c374fd fix: send WS JWT as bare subprotocol name; CSSOM inline styles; open mgmt services on public zone
- websocket.js passes the raw JWT as the Sec-WebSocket-Protocol subprotocol (no 'Bearer ' prefix): subprotocol names must be valid RFC 6455 tokens, and the space in 'Bearer <token>' made the browser reject the constructor with a SyntaxError.
- daemon accepts a JWT-shaped subprotocol plus the legacy 'Bearer <token>' form via _extract_ws_token; unit tests in tests/test_ws_auth.py.
- vdom.js applies inline styles through el.style (CSSOM) instead of setAttribute, which the management-domain CSP (no 'unsafe-inline') blocks.
- install.sh opens http/https/ssh on the public zone alongside WAN setup.
- docs (hoover.md, security.md) updated to match.
2026-08-18 13:36:55 +00:00
mteehan 183904faad fix: seed builtin admin only on empty DB; recover page-load sessions with one refresh
Auth seeding (last-resort guard)
- `_seed_builtin_admin()` in get_db() now skips when
  VACUUM_WALL_SEED_BUILTIN_ADMIN=0 or when the users table already
  contains any user — previously a fresh service start after a non-default
  bootstrap (e.g. --mgmt-user alice) seeded a hard-coded `admin` with an
  unrecoverable random password, shadowing the operator's account
- bootstrap_auth.py sets VACUUM_WALL_SEED_BUILTIN_ADMIN=0: bootstrap
  creates the operator user itself on a fresh install, so exactly one
  account exists and no seeded admin can appear

Frontend (session recovery)
- on page load/restore the in-memory TTL timer is gone, so a valid
  7-day refresh token could sit in sessionStorage while the access token
  is already expired server-side: the session `check` now attempts
  exactly one refresh (POST /api/auth/refresh with the stored refresh
  token) on 401 before treating the session as dead
- extract shared `_doRefresh()` used by both the `check` 401 fallback and
  the `refresh` action (removes the duplicated rotation logic)

Tests
- update seeding tests to the new any-user-present check; add
  test_seed_skipped_when_users_exist, test_seed_skipped_via_env,
  test_bootstrap_flow_creates_exactly_one_user, and the auth-model JS
  test suite (tests/test-auth-model.js)

Docs
- AGENTS.md: document VACUUM_WALL_SEED_BUILTIN_ADMIN
- architecture.md / hoover.md / security.md: describe the bootstrap
  check 401 → one-refresh fallback path
2026-08-18 00:00:09 +00:00
mteehan 0ed275835d fix: auth review fixes — token revocation, WS auth, seeding, and hardening
Refresh/logout and token robustness
- drop the post-rotation refresh_tokens row delete in auth_refresh so
  logout blacklists the current (rotated) refresh token; remove the
  dead _clear_refresh_token_after_rotation helper and clear_active_refresh_token
- reject non-object JWT payloads in _extract_unverified_sub so crafted
  Authorization headers return 401 instead of crashing with 500

SQLite user store
- make builtin-admin seeding idempotent: on a concurrent first start the
  losing seeder re-checks, finds the winner, and returns instead of
  raising IntegrityError
- per-thread sqlite connections + busy_timeout so Flask worker threads
  don't hit cross-thread ProgrammingError / SQLITE_BUSY
- add LogsDirectory + /var/log/vacuum-wall to ReadWritePaths in both
  systemd units so the fallback admin password actually lands on disk

Frontend
- skip apiFetch 401-recovery for public auth endpoints so a failed
  login no longer logs out a valid session
- add /passkeys to the nav (passkey registration was unreachable);
  remove the dead checkWebAuthnCapable export
- drop the CSP-blocked inline WS-URL script and the
  __WS_URL_PLACEHOLDER__ plumbing; the WS URL is derived from location

Daemon / WS
- parse Sec-WebSocket-Protocol manually (web.Request.get_subprotocols
  does not exist in aiohttp 3.13); X-Auth-Token is a custom-nginx
  fallback only — docstring and security docs corrected

Install / system
- bootstrap_auth.py is now idempotent: preserves existing auth config
  and syncs the admin password on re-runs (new reset_password helper)
- WebUI server block renders auth_basic off (the UI is JWT-protected)
- install.sh chown/chmod skips .git to avoid git dubious-ownership
  breakage
- tolerate unreadable /etc/wireguard during system import

Contracts / docs
- create_user returns 409 on duplicate username per docs/api.md
- correct docs/api.md response shapes, docs/security.md blacklist
  cleanup wording + one-refresh-per-user caveat, stale WS-URL
  references, and the .htpasswd description

Tests: +7 regression tests (rotation/logout revocation, crafted-token
401, concurrent seeding); placeholder-substitution tests replaced with
serve-as-is SPA root tests.
2026-08-17 01:45:15 +00:00
mteehan 1980043afd docs: update hoover.md, audit residual storage reads, verify 2026-08-15 08:25:40 +00:00
mteehan 4f74192302 app: use auth model for init, sidebar, login 2026-08-15 02:08:16 +00:00
mteehan 64f3a77411 auth: slim components/auth.js to ceremony helpers + doLogin 2026-08-15 01:54:50 +00:00
mteehan 11a398ce89 ws: decouple reconnection from auth internals, use model 2026-08-15 01:41:09 +00:00
mteehan 382bbd989b api: remove auth helpers, route 401 through auth model 2026-08-15 00:53:05 +00:00
mteehan 5d84710d1a auth: create auth model definition (hoover/auth_model.js) 2026-08-14 23:42:56 +00:00
mteehan 61d95b99a4 model: add onSuccess/onFailure lifecycle hooks to modelRegister 2026-08-14 23:22:39 +00:00
mteehan c7593f8a1e fix: close refresh token session binding bypass
Add require_session parameter to validate_token to enforce session_id
matching for refresh operations. Attacker with stolen refresh token can
no longer bypass session binding by omitting session_id from request.

Also adds backend guard against deleting builtin admin user (was only
blocked at Flask blueprint layer), and removes unused _ALL_RW variable.
2026-08-12 20:06:05 +00:00
mteehan 9ae2cca801 fix: clean up lazy import, dead var, whitespace; document WS session caveat
- Move get_all_credential_counts import to top-level in daemon/handlers/auth.py
- Remove unused PROJECT_DIR in scripts/bootstrap_auth.py
- Fix leading space in api.js tryRefreshToken function
- Document WebSocket session binding limitation in docs/security.md
2026-08-12 19:18:28 +00:00
mteehan 818721ae5c Fix incorrect password log path in _seed_builtin_admin
The warning message referenced 'data/auth.log' but the actual file is
/var/log/vacuum-wall/auth.log. Also add error logging when the write
fails, instead of silently swallowing the OSError.
2026-08-12 18:21:05 +00:00
mteehan 85d8770ba6 fix: address auth subsystem issues from ws-debug review
- lib/auth: make RateLimiter.is_allowed read-only (no dict mutation on read)
- daemon/server: add periodic blacklist_expired cleanup to poll loop (60s interval)
- daemon/server: negotiate only matched Bearer subprotocol on WebSocket connect
- webui/server: rewrite _is_personal_auth with path-prefix matching, cover WebAuthn register routes
- daemon/handlers/auth: eliminate redundant get_user call in auth_update_user
2026-08-12 17:51:09 +00:00
mteehan 0889ef0d08 fix: prevent data loss in update_permissions and token_refresh
- update_permissions: swap to upsert-first-then-delete-stale so a
  failed upsert mid-loop rolls back cleanly, leaving the user's
  permissions intact. Adds Q_DELETE_PERMISSION_SUBSYSTEM for
  targeted removal.

- auth_refresh: generate and persist the new refresh token before
  blacklisting/clearing the old one, so a failure in generate_tokens
  doesn't leave the user locked out with no valid refresh token.
2026-08-12 16:37:06 +00:00
mteehan 6404508519 fix: harden auth with refresh token session binding, logging, and router state
- Add session_id to refresh tokens and enforce it during validation,
  preventing stolen refresh tokens from being usable without the
  originating browser session
- Set router.isAuthenticated via auth:login event after successful
  login (previously only set at page load)
- Add console.warn logging to WS message parse/handler errors
- Improve _refreshPromise error handling in token refresh flow
- Document rate limiter in-memory limitation and CSP connect-src
  same-origin requirement
- Add 3 tests for session-bound refresh token validation
2026-08-12 15:53:17 +00:00
mteehan 76300e281f security: harden builtin admin pwd logging and fix auth token persistence
- Truncate admin password in logs; write full password to data/auth.log (0o600)
- Persist access token in sessionStorage so it survives page reloads
- Simplify tryRefreshToken to use GSAP-style promise deduplication
- Remove spurious POST redirect on 401 during token refresh
- Guard passkey button reference in login finally block
2026-08-12 14:54:07 +00:00
mteehan e01574c67e auth: make session binding optional, enforce WebAuthn credential ownership
- Make session_id parameter optional in validate_token — only enforced when
  provided, allowing WebSocket auth which cannot carry custom headers
- Remove decode_token round-trip from daemon WS handler
- Override WebAuthn registration username from JWT user_ctx to prevent users
  from registering credentials under another user's account
- Frontend no longer sends username for WebAuthn registration
- Redirect to login on 401 after token refresh fails for POST requests
- Remove redundant auth session check from login page load
- Add tests for session_id semantics and WebAuthn ownership guard
2026-08-12 14:16:49 +00:00
mteehan 3654209b78 auth: fix WS session_id extraction and track WebAuthn success/failure
Browsers cannot send custom X-Session-Id header on WebSocket connections,
so decode the token payload to extract session_id. Add WebAuthn
success/failure recording to support rate limiter counter resets.
2026-08-12 04:27:14 +00:00
mteehan 6f728cf853 fix auth: validate logout body, harden WebAuthn, optimize list_users
- Raise ValueError on missing request body in auth_logout
- Add username check in verify_authentication to prevent credential reuse
- Replace N+1 queries in list_users with single JOIN query
2026-08-12 03:58:54 +00:00
mteehan ba0c7bfa9b remove unused token query param fallback from WS auth
The ?token= fallback leaked JWTs in server logs and was never used by
the client, which always sends the token via WebSocket subprotocol
header.
2026-08-12 01:46:00 +00:00
mteehan c64f988ba2 fix: dhcp auto-sync UI refresh, users.js null guard, and login toast escaping 2026-08-12 01:46:00 +00:00
mteehan b69ca330f4 enforce mandatory X-Session-Id header for access token validation
Session binding was bypassable: if the X-Session-Id header was absent,
validate_token skipped the check entirely, allowing a stolen JWT to be
used without the originating session.

Server-side: reject 401 early in Flask middleware and daemon WebSocket
handler when X-Session-Id is missing, before calling validate_token.
Updated validate_token to always enforce session_id matching for access
tokens (refresh tokens are unaffected as they carry no session_id claim).

Frontend: removed dead if (stored.session_id) guards in api.js since
the header is now always required. Added X-Session-Id to logout request
headers and always store session_id on login/refresh.
2026-07-30 22:55:16 +00:00
mteehan 43b44ad340 fix: add probabilistic cleanup to token blacklist
The blacklist table grew indefinitely since cleanup only ran on
password changes and user deletions. Now each blacklist_token()
call has a 2% chance of triggering blacklist_expired() to prune
expired entries. Redundant cleanup calls in update_password()
and delete_user() are removed.
2026-07-29 03:41:15 +00:00
mteehan 48f8d0be18 Auth: rate limiter, WebAuthn domain awareness, misc fixes
- Rate limiter tracks failures only; success resets counter
- Record failures/successes after password verification, not before
- WebAuthn rp_id/origin resolved dynamically from request domain
- Management domains auto-discovered from nginx backend config
- All WebAuthn operations validate domain against management list
- Add GET /api/auth/webauthn/capable endpoint for frontend checks
- Frontend checkWebAuthnCapable() function for domain-gated UI
- Timing side-channel fix: pre-compute dummy hash at module load
- Builtin admin seeded with random password (logged at WARNING)
- Logout handler returns consistent response shape
2026-07-29 02:48:53 +00:00
mteehan 6d30f1387e fix: indentation in frontend handlers and restore dhcp auto-sync toast 2026-07-28 19:32:17 +00:00
mteehan 8ae60ab8cf fix: harden auth and fix frontend issues
- Add builtin admin user with full access, immutable permissions (lib/db.py, lib/auth_users.py, webui/static/pages/users.js)
- Fix passkeys TypeError on string throws (webui/static/pages/passkeys.js)
- Add zero-permission warning in create user modal (webui/static/pages/users.js)
- Restore readonly on proxy paths textarea (webui/static/pages/proxy.js)
- Mask credential ownership errors to prevent enumeration (lib/webauthn.py, tests/test_auth.py)
2026-07-28 18:52:03 +00:00
mteehan a82578f342 fix: invalidate tokens on permission change (medium), optimize create_user query, fix ws reconnect race
- update_permissions now calls blacklist_active_refresh_token and
  rotate_user_secret to immediately invalidate stale tokens
- create_user uses returned id from tx.run_one instead of redundant SELECT
- websocket reconnect explicitly closes old connection after token refresh
  to prevent onclose handler race condition
2026-07-28 18:02:14 +00:00
mteehan 8bb3619ddc refactor: extract shared utilities and standardize page patterns
- Add fmtBytes() and csvToArr() helpers to hoover/helpers.js
- Replace inline async patterns with ActionButton/ConfirmDelete in wireguard.js
- Convert addDomain/editDomain to QuickModal + apiSubmit in proxy.js
- Convert settingsModal handlers to formAction in certs.js
- Remove redundant synced handling from dhcp.js apply button
- Add onComplete callback to ConfirmDelete (fixes users.js onRefresh bug)
- Fix passkeys.js ActionCell/Table usage (invalid component API)
- Remove duplicate fmtBytes from dashboard.js
2026-07-28 17:32:51 +00:00
mteehan 244576b8eb fix: render Add passkey button on empty state
The Empty component ignores children, so the button was silently dropped
when using <Empty> as an htm wrapper. Inline the card structure instead.
2026-07-28 16:45:34 +00:00
mteehan 3de82e3b9b Remove query-string cache-busting from static assets
Drop ?v=N version pins from all JS imports and HTML <link>/<script> tags.
Cache invalidation is now handled solely by server-side cache-control headers.
Update docs and AGENTS.md accordingly.
2026-07-28 13:50:22 +00:00
mteehan c943d17bb3 Fix typo: 'PassKey' -> 'Passkey' in toast message 2026-07-28 13:44:52 +00:00
mteehan f77473c13c fix: missing brace in users.js, ws double-increment, b64url stack overflow, docs storage 2026-07-28 13:17:12 +00:00
mteehan ca27ea5522 feature: framework-level abort handling for page lifecycle
component.js now creates an AbortController for each page mount, passing
it to load(). On unmount, the controller is aborted to cancel in-flight
requests that would otherwise mutate unmounted state.

Page load functions consistently pass the signal to apiFetch and guard
state mutations with abort checks. This eliminates the need for per-page
abortController boilerplate and prevents stale errors from appearing on
rapid navigation.

Users page now guards catch block and loading state cleanup against
aborted requests, matching passkeys.js pattern.
2026-07-28 02:52:28 +00:00
mteehan d52a0fad12 fix: check tryRefreshToken result in WS reconnect, clear permissions on logout
The token refresh callback in websocket.js ignored its ok parameter, causing
an infinite reconnect loop when the server rejected the refresh attempt. On
failure, redirect to login instead of reconnecting with a cleared token.

clearAuthTokens() now also removes vw:permissions from sessionStorage to
prevent stale permissions from persisting across logout/login cycles. Also
removed duplicate vw:user removeItem call.
2026-07-28 02:44:09 +00:00
mteehan c4a10c7129 refactor: switch auth data from localStorage to sessionStorage
Aligns user and permissions storage with the existing sessionStorage-based
token model. Eliminates the dual-write pattern and stale cross-session data.
2026-07-28 02:16:24 +00:00
mteehan 739253b2e5 fix: WS reconnection deadlock, remove redundant try/except, fix docs
- websocket.js: schedule reconnect backoff when token refresh fails,
  otherwise WebSocket stays dead after 3+ disconnects with failed refresh
- daemon/handlers/auth.py: remove two redundant try/except ValueError: raise
  blocks in webauthn register/authenticate finish handlers
- docs/api.md: mark permissions as optional in Create User endpoint
2026-07-28 01:44:10 +00:00
mteehan 358573567d fix: dual-key rate limiting for auth + websocket reconnect guard
- Pass client IP (X-Real-IP header) through Flask to daemon for both
  password login and WebAuthn authenticate-finish endpoints
- Rate limiter now checks both IP and username buckets: IP layer catches
  enumeration/brute-force attacks across multiple usernames; username
  layer protects against single-account targeting from multiple IPs
- Add _wsRefreshing flag to prevent double-scheduling reconnect when
  onclose fires during token refresh; simplify async IIFE to .then()/.catch()
- Reset _wsRefreshing on websocket onopen for safety
2026-07-28 00:54:48 +00:00
mteehan 76cd219050 fix: harden retry JSON parsing and exempt personal auth routes
webui/static/hoover/api.js
  Guard retryRes.json() with .catch(() => null) so non-JSON
  responses (e.g. nginx 502/503) don't throw and lose the
  actual status code. Falls back to 'HTTP <status>' error string.

lib/db_sqlite.py
  Replace unsafe sql.split(';') loop with conn.executescript()
  which properly handles semicolons inside string literals.

webui/server.py
  Add _AUTH_PERSONAL set and _is_personal_auth() so personal
  auth operations (session, password, logout, webauthn creds)
  skip subsystem permission checks. Users with only firewall:read
  can now manage their own credentials without needing auth:rw.
2026-07-27 20:38:50 +00:00
mteehan e48ba72b81 fix: 401-401 retry on non-401, move login DOM bindings into page lifecycle, add abort support 2026-07-27 19:46:30 +00:00
mteehan cc5679a1cd fix: deduplicate token refresh, serialize concurrent attempts, clean up logout path 2026-07-27 19:15:27 +00:00
mteehan ca110c321d style: format docs, fix user_permissions variable scoping in auth middleware
Apply ruff line-wrapping formatting to docs and test files.
Clarify auth middleware: extract user_permissions once before
subsystem check, removing conditional variable scoping.
2026-07-27 18:37:11 +00:00
wall d4213fb93b fix: auth reconnection loop, duplicate login listeners, modal double-disable
- websocket: clear tokens on refresh failure to prevent infinite 401 loop
- api: write vw:user to sessionStorage on refresh for consistency with WS
- api: remove vw:user from sessionStorage in clearAuthTokens
- login: guard listener setup with flags to prevent duplicate attachment
- modal: skip inline button disable when handler uses processing state
- users: remove unused requestUpdate import
2026-07-24 04:02:44 +00:00
mteehan edaf16a433 fix: remove dead auth_refresh code, fix logout storage, align bootstrap TTL, add hash rehash, tighten CSP
- Remove duplicate dead code in daemon/handlers/auth.py (auth_refresh)
- Fix logout reading refresh token from localStorage instead of sessionStorage
- Align bootstrap auth config access_token_ttl (900 -> 300) with hardened default
- Add password hash rehash check on successful login (needs_rehash was unused)
- Remove 'unsafe-inline' from CSP style-src (all styles are applied via JS DOM API)
2026-07-24 03:09:25 +00:00
mteehan a365059976 security: harden JWT auth with session binding, CSP headers, and sessionStorage
- Reduce access_token_ttl from 900s to 300s (5 min) to shrink XSS exploit window
- Add session_id claim to JWT tokens tied to browser session (X-Session-Id header)
- Flask middleware validates session_id matches header on every request
- CSP headers: default-src/script-src 'self', no unsafe-inline/eval, frame-ancestors none
- X-Content-Type-Options: nosniff on all responses
- Move refresh token from localStorage to sessionStorage (tab-scoped, cleared on close)
- Timing-safe password verification (dummy Argon2id for unknown users)
- WebSocket auth also validates session_id header
- Add 5 session_id tests and 3 CSP header tests
2026-07-24 03:09:07 +00:00
mteehan 56b200d233 feat: add auth subsystem with WebAuthn passkeys support
New modules: lib/auth, lib/auth_users, lib/db, lib/db_sqlite, lib/password,
lib/webauthn, daemon/handlers/auth, scripts/bootstrap_auth, tests/test_auth

Frontend: webui/api/auth, hoover/components/auth, pages/login, passkeys, users

Updates: daemon/iface and server, lib/common and nginx, pyproject.toml deps,
install script, server.py, app.js, and websocket/api clients
2026-07-24 01:21:39 +00:00
mteehan 04417cf05c WireGuard access classes, firewall nftables fixes, network sync event refactor
- WireGuard: refactor to multi-interface 'access classes' model; extract config
  generation and helpers into lib/wireguard.py; add per-class up/down endpoints
  and API routes; update UI with class management pages and QR code component
- Firewall: fix zone creation with --new-zone before --set-target; skip
  masquerade on public zone; add masquerade propagation for nftables backend
  so NAT works when internal zones exit via public
- Network: rename sync event subsystem 'network' -> 'networkd'; always stamp
  config hash even when deployment fails (fixes pending-changes detection)
- DHCP: add new API endpoint and update frontend page
- State/Sync: update state collectors and sync buses for new subsystems
- Docs: update API and config documentation for new endpoints and schemas
2026-07-20 03:57:16 +00:00
mteehan dadabd7954 feat: add system metrics dashboard with resource monitoring
- Add system metrics endpoint (CPU load, memory, swap, network traffic)
- Collect metrics from /proc and /sys (no subprocess required)
- Overhaul dashboard to pull from per-subsystem models
- Remove deprecated /status/all monolithic endpoint
- Improve networkd import to handle optional priority prefix
- Fix CSS duplicate .grid-4 rule and unused dashboard imports
2026-07-15 00:24:43 +00:00
mteehan c21639b7f1 docs: add comprehensive docstrings and inline comments
Add docstrings to all handler functions in daemon/handlers/firewall.py, covering
params, return values, and raised exceptions. Add inline comments to
_config_apply() reconciliation steps and the request body merge order.

Add docstrings across lib/ modules for emit helpers (_emit_str, _emit_int, etc.),
volatile stripping logic, two-layer diff strategy, sync event dispatch, and all
cross-subsystem sync subscribers (DnsToFirewall, WgToFirewall, FirewallToDhcp,
NetworkToAllSync).

Document WireGuard/networkd config parsers and key-value mappers in
system_import.py. Add docstrings to _ep(), Registry.decorator,
setup_logging, and _replace helper across daemon/ and lib/.
2026-07-13 17:26:45 +00:00
mteehan 2e49dec633 feat: auto-generate self-signed certs, hoover modal processing guard, refactor backends/proxy pages 2026-07-13 14:30:35 +00:00
mteehan 05524f3756 fix: critical bugs + security hardening
Phase 1 (critical bugs):
- Fix firewall import string-to-list bug (system_import.py)
- Add rich rules removal in firewall config apply (handlers/firewall.py)

Phase 2 (security hardening):
- Restrict sudo wildcards to specific paths (sudoers.d/vacuum-walld)
- Fix TOCTOU: use /run/vacuum-wall/ for temp files (nginx, dnsmasq, network handlers)
- Remove unnecessary sudo from wg genkey/pubkey (handlers/wireguard.py)

Phase 3 (validation):
- Validate poll intervals > 0 (daemon/server.py)
- Restrict sysctl to whitelisted parameters (handlers/network.py)

Phase 4 (defensive programming):
- Enforce shell=False in run() and run_proc() (lib/common.py)
- Track issuance tasks for graceful shutdown (handlers/acme.py)
- Add nginx template marker consistency tests (tests/test_system_import.py)
2026-07-11 12:21:36 +00:00
mteehan 803258cf18 dhcp: auto-populate gateway from interface IP for DHCP ranges
Add get_interface_ip() helper to resolve an interface's IPv4 address
via 'ip -o addr show'.  Use it to back-propagate gateway into DHCP
ranges so clients receive their default route.

- set_dhcp_range() resolves gateway: explicit > existing range > iface IP
- DnsToFirewallSync and FirewallToDhcpSync sync ensure gateways are set
- Remove automatic masquerade toggle from DnsToFirewallSync
- Fix dnsmasq lease file path to /var/lib/misc/dnsmasq.leases
- Rename lease state field expires_at -> expires (ISO string)
- Add 'ip -o addr show' to sudo whitelist
2026-07-09 00:58:09 +00:00
mteehan 5135de0921 feat: add system config import, refactor install script and nginx auth
- lib/system_import: new module to import system configs into JSON at daemon startup
- daemon/server.py: call import_all() during startup for config reconciliation
- daemon/handlers/nginx.py: simplify add_domain auth handling, remove duplicate code
- scripts/install.sh: replace inline Python setup with curl-based daemon API calls; apply IP forwarding at runtime
- hoover: bump internal asset versions to v=8
- pages: bump asset versions to v=9
2026-07-08 02:20:28 +00:00
mteehan fb39af126a docs: update documentation and project structure
- Update AGENTS.md, README.md, and docs/* with revisions
- Refactor lib/acme.py and lib/state.py
- Add tests for acme module
- Remove install.sh and restart-services.sh (moved to scripts/)
- Normalize vendor files (acme.sh, htm.js)
2026-07-02 14:41:02 +00:00
mteehan b4d13c4bd5 Add restart-services script to scripts/ directory 2026-07-02 14:39:35 +00:00
mteehan 20266e4a2f rename vendor files to put extension after version string
Flask's send_file uses mimetypes.guess_type() which needs the file extension
at the end of the path. Previously files were named acme.sh-3.1.3 and
htm.js-3.1.1 — now acme-3.1.3.sh and htm-3.1.1.js.

Also: update-vendor.sh now owns ACME_HOME installation, removing duplicate
logic from install.sh.
2026-07-02 04:01:52 +00:00
mteehan 8c13ad55ce Add update-vendor.sh symlink support, unify install.sh vendor flow
- update-vendor.sh now creates webui/vendor symlinks (htm.js)
- install.sh calls update-vendor.sh after package install
- Add vendor/.empty and webui/vendor/.empty as directory placeholders in git
2026-07-01 00:55:03 +00:00
mteehan 575cf06a4b hoover: move ToastContainer to separate module, add reactive re-render, bump module cache v8 2026-06-30 04:01:41 +00:00
mteehan 9088f34345 sync: add cross-subsystem event bus for config consistency
Add EventBus with loop guards to keep firewall, dnsmasq, wireguard,
and network configs consistent. Handlers emit SyncEvent after mutations;
subscribers compute diffs and write JSON without manual cascade loops.
2026-06-30 01:18:44 +00:00
mteehan 348bbfbca6 dhcp: track pending config changes with hash, update UI button 2026-06-28 16:26:32 +00:00
mteehan baa441fa13 dhcp: auto-populate IP range when selecting interface
- QuickModal: add postRender callback for DOM-level initialization
- dhcp: derive start/end from interface CIDR on change
2026-06-28 13:04:51 +00:00
mteehan 391466664e Update status checks and add optgroup support to form modals
- modal.js: add optgroup support for select options in formModal
- dhcp.js: use zone-based interface selector for DHCP ranges
- dashboard.js, wireguard.js: use status.up for state checks
- proxy.js: use inner.querySelector for modal field lookup
2026-06-28 12:47:59 +00:00
mteehan d8d8425340 docs: update API conflict status, ACME config, request flow, deployment, and security sudoers 2026-06-28 01:58:56 +00:00
mteehan 25a1943fce Optimize firewall state collection and improve daemon shutdown
- Replace per-zone --list-all calls with single --list-all-zones in _collect_firewall
- Add _parse_all_zones_output() parser with rich rules/rich-rules normalization
- Convert daemon shutdown to async with proper runner cleanup and socket unlink
- Add TimeoutStopSec=15 to vacuum-walld.service for graceful stop
- Fix exception handling in _collect_dnsmasq
- Remove management badge from proxy path rows
2026-06-28 00:54:01 +00:00
mteehan 80dd4e3272 proxy: refactor modals and add cert selector options 2026-06-27 23:59:12 +00:00
151 changed files with 30222 additions and 13703 deletions
+6
View File
@@ -8,6 +8,12 @@ __pycache__/
# Package build
*.egg-info/
# vendor dirs
vendor/*
!vendor/.empty
webui/static/vendor/*
!webui/static/vendor/.empty
# Tool caches
.pytest_cache/
.ruff_cache/
+86 -56
View File
@@ -8,7 +8,7 @@ Deploys on Debian 13 (trixie). Serves from repo root by default.
## Architecture
```
Client ──→ nginx (SSL + basic auth) ──→ Flask (127.0.0.1:9090)
Client ──→ nginx (TLS; basic auth on basic-authed proxy domains only) ──→ Flask (127.0.0.1:9090)
Flask ──→ daemon/client.py (Unix socket) ──→ vacuum-walld (aiohttp, daemon.sock)
vacuum-walld ──→ daemon/handlers/*.py ──→ sudo <cmd> ──→ system service
```
@@ -16,6 +16,15 @@ vacuum-walld ──→ daemon/handlers/*.py ──→ sudo <cmd> ──→ syste
Blueprints are thin proxies — they never call `lib/` directly. All operations flow
through the daemon client over a Unix socket.
**Authentication:** the management UI gets **no** nginx-level `auth_basic` — the mgmt
server block's `location /` is a bare proxy and `/ws` is `auth_basic off`. Management
auth is the Flask-layer JWT middleware (`POST /api/auth/login`
`Authorization: Bearer <token>`; public paths: static files, `/vendor/`, auth endpoints)
plus the daemon WS handshake (raw JWT as the `Sec-WebSocket-Protocol` subprotocol).
Basic auth (`.htpasswd`) renders **only** for proxy domains whose
`config/nginx/config.json` has an `auth` block — never for the management domain
(see `docs/security.md`, "Management Interface").
### Two-User Model with Shared Group
- **`vacuum-walld`** (daemon user): runs the privileged background daemon with `NOPASSWD sudo` whitelist (`/etc/sudoers.d/vacuum-walld`). Owns socket. Primary group is the WebUI user's primary group. Daemon user name is derived: `USER_NAME` + `d`.
@@ -24,74 +33,78 @@ through the daemon client over a Unix socket.
### Code Layout
- `webui/server.py` — Flask app entry point. **Only** file that creates the `app`. SPA catch-all renders `index.html` with server-side `__WS_URL_PLACEHOLDER__` substitution (no Jinja).
- `webui/server.py` — Flask app entry point. **Only** file that creates the `app`. SPA root route (`/`) serves `index.html` (no templating). All other paths return 404.
- `webui/api/*.py` — Flask blueprints, one per subsystem. Routes prefix `/api/<subsystem>/`. All call `daemon.client` instead of `lib/` directly.
- `webui/api/common.py` — Shared `_ok()` / `_error()` response helpers used by all blueprints.
- `daemon/server.py` — aiohttp server, route registry, batch routing, WebSocket broadcast, state refresh.
- `daemon/server.py` — aiohttp server, route registry, batch routing, WebSocket broadcast, state refresh, periodic polling.
- `daemon/client.py` — Sync HTTP client over Unix socket using `requests_unixsocket.Session`.
- `daemon/iface.py`**Single source of truth** for all daemon API endpoints. Every endpoint is a frozen `(method, path)` tuple. Renaming an endpoint here auto-updates both server registry and client calls. All blueprints and handlers import from here.
- `daemon/handlers/*.py` — Privileged operation handlers (all `sudo` calls live here).
- `lib/state.py` — In-memory state store with per-subsystem collectors. Populated at daemon startup, refreshed on request. Backs WebSocket versioning/broadcast.
- `lib/common.py`Shared utilities: `run()`, `run_proc()`, `load_json()`, `save_json()`, `deep_merge()`, `ensure_dirs()`, `validate_interface_name()`. All `lib/` modules use these instead of defining local helpers.
- `lib/logging.py` — Logging setup used by both webui and daemon.
- `daemon/iface.py`**Single source of truth** for all daemon API endpoints. Every endpoint is a frozen `(method, path)` tuple. Renaming here auto-updates both server registry and client calls.
- `daemon/handlers/*.py` — Privileged operation handlers. All mutating `sudo` calls live here.
- `daemon/collectors/` — Per-subsystem state collectors (7 modules: firewall, dnsmasq, nginx, acme, wireguard, networkd, system). Read-only `sudo` queries that populate `lib.state`. Imported for their registration side-effect; `daemon/server.py` imports the package before the first `populate()`.
- `lib/state.py`In-memory state store with per-subsystem collectors. Populated at daemon startup, refreshed on mutation/poll. Backs the WebSocket push stream: `get_snapshot()` (full state on WS connect), `poll()` two-layer diff (structural `versions` broadcast vs volatile-only `tick` broadcast, per-subsystem, each carrying the full subsystem data), `register_volatile(subsystem, keys)` to mark volatile fields, `get_versions()`/`bump()`. Per-subsystem poll intervals via `_DEFAULT_POLL_INTERVALS` (system 1s, firewall 30s, wireguard/dnsmasq/networkd 10s, nginx 60s, acme 300s).
- `lib/common.py` — Shared utilities: `run()`, `run_proc()`, `load_json()`, `save_json()`, `deep_merge()`, `ensure_dirs()`, `config_hash()`, `validate_interface_name()`, plus the apply-bookkeeping helpers `stamp_applied()` / `strip_apply_meta()` / `compute_pending()` / `deep_diff()` / `revert_to_applied()` (the `_last_applied_hash` / `_last_applied_config` keys every config-backed subsystem uses for pending-change detection and cancel-all).
- `lib/logging.py` — Logging setup used by both webui and daemon. Reads `VACUUM_WALL_LOG_LEVEL`.
- `lib/sync.py` — Cross-subsystem sync event bus (in-process pub/sub); handlers emit events on mutation and subscribers refresh affected subsystems.
- `lib/system_import.py` — Startup system-config import/reconcile: parses native config sources and merges them into the declarative JSON on daemon start.
- `lib/bootstrap.py` — Daemon-startup filesystem bootstrap, run **after** `system_import.import_all()` (which must see absent config files to adopt live state on first start) and before the first state collection: creates the runtime `config/`+`data/` directories and persists the one-shot nginx legacy-format migration. Never creates config files (reads stay pure; files appear on first `save_config`).
- `lib/schema.py` — TypedDict state schemas for the per-subsystem state payloads.
- `lib/*.py` — Backend modules (parsing, config, shared logic). Full type hints and `__all__` exports. No sudo calls.
- `vendor/` — Vendored scripts and JS libraries (`acme.sh`, `htmx`, `json-enc`).
- `vendor/` — Vendored scripts and JS libraries (`acme.sh`, `htm`).
- `data/` — Runtime artifacts (generated .confs, `.htpasswd`, ACME certs, firewall backup, dnsmasq fragments).
- `config/<subsystem>/config.json` — Declarative JSON configs (source of truth). Generated `.conf` in `data/nginx/sites-enabled/`. Certs in `data/acme/`.
- `config/<subsystem>/config.json` — Declarative JSON configs (source of truth).
- `system/` — System file templates. `systemd/` (units installed to `/etc/systemd/system/`), `sudoers.d/`, `nginx/`.
Project uses `.venv`. Install deps with `pip install -e .` (from `pyproject.toml`). `__init__.py` files in `webui/`, `lib/`, and `daemon/` are intentionally empty.
Project uses `.venv`. Install deps with `pip install -e .` (from `pyproject.toml`).
`__init__.py` files in `webui/`, `lib/`, and `daemon/` are intentionally empty.
### Frontend (hoover)
Custom reactive SPA framework at `webui/static/hoover/`. See `docs/hoover.md` for full API reference.
Conventions:
- All imports from `/static/hoover/index.js` (barrel export of reactivity, VDOM, router, API, components).
- Pages in `webui/static/pages/` export `definePage({ init, subscribe, load, render })` as default.
- All imports from `/static/hoover/index.js` (barrel export).
- Pages in `webui/static/pages/` export `definePage({ init, subscribe, load, render })`.
- Bootstrap: `webui/static/app.js` mounts two render roots (`#sidebar`, `#main`), then `connect()` for WS.
- `h()` builds VNodes; `html` tag (from htm) enables JSX-like templates; `#comp` + `hComp()` for component lifecycle; `key` for keyed diff.
- Events: `h()` uses `on:click` prefix. `html` templates use camelCase `onClick` (adapter translates to `on:click`).
- `h()` builds VNodes with `on:click` prefix. `html` tag (htm) templates use camelCase `onClick` (adapter translates).
- State always has `loading`, `refreshing`, `error` plus data. `load()` receives `(state, abortController, entry)`.
- `openModal` + `formModal` for dialogs; `apiSubmit()` for form submission. `ToastContainer()` in main root.
- No build step — ES modules served raw. Assets versioned via `?v=N` query string.
- `openModal` + `formModal` for dialogs; `apiSubmit()` for form submission.
- No build step — ES modules served raw. Cache controlled via HTTP headers. For the management domain, nginx serves `/static/` directly from `webui/static/` (generated `location /static/` alias with `no-cache` + ETag revalidation); Flask's static route is the dev-mode fallback.
### Daemon Endpoints
- Unix socket at `data/daemon.sock` (configurable via `VACUUM_WALLD_SOCKET` env var)
- WebSocket at `127.0.0.1:9091` (configurable via `VACUUM_WALLD_WS_PORT`) for real-time state change notifications
- Can be started as `python -m daemon.server` or via the `vacuum-walld` console script
- Unix socket at `data/daemon.sock` (configurable via `VACUUM_WALLD_SOCKET`)
- WebSocket at `127.0.0.1:9091` (configurable via `VACUUM_WALLD_WS_PORT`) for real-time state streaming: full `snapshot` on connect, then per-subsystem data-carrying `versions` (structural) / `tick` (volatile-only) deltas. The client patches reactive models in place via `modelSet()` — no HTTP round-trip for auto-refresh.
- Periodic polling per subsystem via `lib.state._DEFAULT_POLL_INTERVALS`, overridable with `VACUUM_WALL_POLL_INTERVALS` env var (format `subsystem:seconds,subsystem:seconds`)
- Start as `python -m daemon.server` or via the `vacuum-walld` console script
## Environment Variables
- `VACUUM_WALL_DEV` — dev mode flag; when set, disables aggressive static asset caching
- `VACUUM_WALLD_SOCKET` — override daemon socket path
- `VACUUM_WALL_DEV` — dev mode flag; disables aggressive static asset caching
- `VACUUM_WALL_LOG_LEVEL` — log level (default `INFO`)
- `VACUUM_WALLD_SOCKET` — override daemon socket path (default `data/daemon.sock`)
- `VACUUM_WALLD_WS_PORT` — override WebSocket port (default `9091`)
## Deployment
`install.sh` installs only system components and configures them; the project serves from the repo root by default. Options via env vars or CLI flags (CLI takes precedence). Set `INSTALL_DIR` or `--path` to override. Use `--dev` to auto-detect repo owner as service user; non-dev requires `--user`.
All Python modules use `Path(__file__).resolve().parent.parent` for `PROJECT_DIR` — no hardcoded paths. ACME certs at `PROJECT_DIR/data/acme/`.
### Service Start Order
`firewalld``avahi-daemon``dnsmasq``vacuum-walld``vacuum-wall`
- `VACUUM_WALL_POLL_INTERVALS` — override poll intervals, e.g. `firewall:60,wireguard:5`
- `VACUUM_WALL_EXTERNAL_IP_URL` — custom URL for external IP detection (acme handler)
- `VACUUM_WALL_SEED_BUILTIN_ADMIN` — set to `0` to skip the last-resort builtin admin seed in `get_db()`. The seed only runs on a completely empty DB (no users); `scripts/bootstrap_auth.py` always sets this since bootstrap creates the operator user itself.
## Local Dev
```bash
.venv/bin/python webui/server.py # binds 127.0.0.1:9090
# Setup
python3 -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"
bash scripts/update-vendor.sh # fetches acme.sh + htm.js
# Start (Flask only, binds 127.0.0.1:9090)
.venv/bin/python webui/server.py
```
Reload running Flask via SIGHUP (auto-reloads `webui.*` and `lib.*` modules, then SIGTERM restart).
In production: systemd units run with `NoNewPrivileges`, `ProtectSystem=strict`, loopback-only networking.
## Blueprint / Handler / lib Mapping
## Blueprint ↔ Handler ↔ lib Mapping
| Blueprint | URL Prefix | Handler Module | lib Module |
|-----------------------|---------------------|--------------------------|-----------------|
| Blueprint | URL Prefix | Handler | lib Module |
|-----------------------|---------------------|----------------------------|-----------------|
| `webui/api/firewall` | `/api/firewall/` | `daemon/handlers/firewall` | `lib.firewall` |
| `webui/api/dhcp` | `/api/dhcp/` | `daemon/handlers/dnsmasq` | `lib.dnsmasq` |
| `webui/api/proxy` | `/api/proxy/` | `daemon/handlers/nginx` | `lib.nginx` |
@@ -99,6 +112,8 @@ In production: systemd units run with `NoNewPrivileges`, `ProtectSystem=strict`,
| `webui/api/wireguard` | `/api/wireguard/` | `daemon/handlers/wireguard`| `lib.wireguard` |
| `webui/api/network` | `/api/network/` | `daemon/handlers/network` | `lib.network` |
| `webui/api/logs` | `/api/logs/` | `daemon/handlers/logs` | — |
| `webui/api/status` | `/api/status/` | `daemon/handlers/status` | — |
| `webui/api/auth` | `/api/auth/` | `daemon/handlers/auth` | `lib.auth` / `lib.auth_users` |
## Privileged Operations
@@ -109,22 +124,44 @@ In production: systemd units run with `NoNewPrivileges`, `ProtectSystem=strict`,
Pattern for mutations: write JSON → render native config → `sudo <cmd>` to apply.
Adding a new privileged command requires a sudoers entry **and** the `daemon/handlers/` code.
**Config reads are pure.** Every `lib/<subsystem>.get_config()` is a side-effect-free
read (returns in-memory defaults when the file is missing; nginx applies its
legacy-format migration in memory). State collectors therefore never write to
disk — filesystem setup (runtime dirs, one-shot nginx migration) happens once at
daemon startup in `lib/bootstrap.py` (after `system_import.import_all()`).
**Firewall interface-coverage invariant.** Every network-managed interface
(`lo`/`wg*` excluded) must be covered by a zone in `config/firewall/config.json`
or listed under the top-level `unmanaged` key. The config is the source of truth
for zone interfaces (an omitted `interfaces` key = empty list; no hands-off
zones). Enforced at save time (`POST`/`PATCH /firewall/config` → 400) and apply
time (`POST /firewall/config/apply` → 409, `force: true` overrides) via the pure
`lib.firewall.validate_coverage()`. Live drift is advisory only
(`uncovered_interfaces` state field). See `docs/config.md`.
## API Response Contract
- Success: `{"ok": true, "data": <value>}` helper `_ok(data)` from `webui.api.common` (Flask) or `ok(data)` from `daemon.server` (aiohttp).
- Error: `{"ok": false, "error": "msg"}` helper `_error(msg, code=400)` from `webui.api.common` or `error(msg, code)` from `daemon.server`.
- `acme.issue()` / `acme.renew()` raise `RuntimeError` on failure — API layer wraps in try/except.
- HTTP codes: `400` bad request, `404` not found, `500` internal failure.
- Success: `{"ok": true, "data": <value>}``_ok(data)` (Flask) or `ok(data)` (aiohttp)
- Error: `{"ok": false, "error": "msg"}``_error(msg, code)` (Flask) or `error(msg, code)` (aiohttp)
- `acme.issue()` / `acme.renew()` raise `RuntimeError` on failure — API layer wraps in try/except
- HTTP codes: `400` bad request, `404` not found, `409` conflict, `500` internal failure
## Deploy
`install.sh` is the single deploy script. Run as root. Only `MGMT_PASS` is strictly required; `MGMT_DOMAIN` is auto-detected from hostname.
`scripts/install.sh` is the single deploy script. Run as root. CLI flags take precedence over env vars.
`MGMT_PASS` is strictly required — it is the **SQLite DB** password for the initial `admin`
user (full `rw` on all subsystems; default username `admin`), **not** an nginx htpasswd.
`MGMT_DOMAIN` auto-detected from hostname.
### Service Start Order
`firewalld``avahi-daemon``dnsmasq``vacuum-walld``vacuum-wall`
## Lint and Tests
**Linter / formatter:** Ruff (`ruff check` + `ruff format`). Config in `pyproject.toml` under `[tool.ruff]`.
**Linter / formatter:** Ruff (`ruff check` + `ruff format`). Config in `pyproject.toml`.
**Tests:** pytest in `tests/`. Tests mock out subprocess calls — no system services required.
**Tests:** pytest in `tests/` (28 Python + 9 JS test files). All subprocess calls are mocked — no system services required.
```bash
.venv/bin/ruff check lib/ webui/ tests/ # lint
@@ -132,24 +169,17 @@ Adding a new privileged command requires a sudoers entry **and** the `daemon/han
.venv/bin/python -m pytest tests/ -v # test
```
Install dev tooling with `pip install -e ".[dev]"`.
## Docs
`docs/` contains the authoritative reference for each subsystem. **Before reasoning about any subsystem**, read the relevant doc(s) below.
`docs/` contains the authoritative reference for each subsystem. Read the relevant doc before reasoning about a subsystem.
| Doc | Contents |
|-----|----------|
| `docs/architecture.md` | Request flow, subsystem communication, two-user model, zone model, state management |
| `docs/security.md` | Privilege model, sudo whitelist, systemd hardening, TLS config, zone trust levels |
| `docs/deployment.md` | Install script options, what install.sh does, post-install setup, troubleshooting |
| `docs/deployment.md` | Install script options, what scripts/install.sh does, post-install setup, troubleshooting |
| `docs/config.md` | JSON schema for each subsystem config (dnsmasq, nginx, wireguard, cert types) |
| `docs/api.md` | REST API endpoint reference, request/response contracts, route patterns |
| `docs/hoover.md` | Custom frontend framework API reference |
| `docs/overview.md` | Subsystem summaries, tech stack, complete project directory tree |
## Important Rules
1. Ask, don't assume. If something is unclear, ask before writing a single line.
2. Simplest solution first. Always implement the simplest thing that could work.
3. Don't touch unrelated code. If a file or function is not directly part of the current task, do not modify it.
4. Flag uncertainty explicitly. If you are not confident about an approach or technical detail, say so.
| `docs/state-model.md` | Per-subsystem state schema, the `versions`/`tick` two-layer diff, and the pending-changes model |
| `docs/overview.md` | Subsystem summaries, tech stack, complete project directory tree |
+25 -16
View File
@@ -7,7 +7,7 @@ A zone-based firewall appliance with a built-in SSL reverse proxy. Combines fire
- Debian 13 (trixie) target platform
- Python 3.13+, Flask 3.x web UI
- firewalld (nftables backend), dnsmasq, nginx, WireGuard
- acme.sh for ACME certificates (ZeroSSL)
- acme.sh for ACME certificates (CA is config-driven; code default Let's Encrypt)
---
@@ -28,13 +28,13 @@ MGMT_DOMAIN=wall.example.com \
MGMT_PASS="strongpassword" \
MGMT_USER="admin" \
ACME_EMAIL="admin@example.com" \
bash install.sh
bash scripts/install.sh
```
### Install (Development)
```bash
./install.sh --dev --mgmt-pass strongpassword --acme-email "admin@example.com"
./scripts/install.sh --dev --mgmt-pass strongpassword --acme-email "admin@example.com"
```
`--dev` auto-detects the repo's file owner as the service user, skips the safety warning about running as a regular user, and keeps file ownership dev-friendly.
@@ -42,9 +42,9 @@ bash install.sh
| Flag | Env Var | Required | Description |
|---|---|---|---|
| -- | `MGMT_DOMAIN` | No | Public domain for the management WebUI (auto-detected as `hostname.local`) |
| `--mgmt-pass` | `MGMT_PASS` | Yes | HTTP basic auth password for the WebUI |
| `--mgmt-pass` | `MGMT_PASS` | Yes | SQLite DB password for the initial `admin` user (full `rw` on all subsystems) — not an nginx htpasswd |
| `--mgmt-user` | `MGMT_USER` | No | WebUI username (defaults to `admin`) |
| `--acme-email` | `ACME_EMAIL` | Yes | ACME registration email (ZeroSSL by default) |
| `--acme-email` | `ACME_EMAIL` | Yes | ACME registration email (CA is config-driven; code default Let's Encrypt) |
| `--user, -u` | `USER_NAME` | No | System user for service (default: `vacuum-wall`) |
| `--path, -p` | `INSTALL_DIR` | No | Install directory (default: repo root) |
| `--dev` | -- | No | Auto-detect repo owner as service user, skip safety warning |
@@ -52,7 +52,7 @@ bash install.sh
| `--wan-iface` | `WAN_IFACE` | No | WAN interface (auto-detected) |
| `--lan-ifaces` | `LAN_IFACES` | No | LAN interfaces, comma-separated (auto-detected) |
CLI flags take precedence over environment variables. Run `./install.sh --help` for full usage.
CLI flags take precedence over environment variables. Run `./scripts/install.sh --help` for full usage.
After installation, access the WebUI at `https://<MGMT_DOMAIN>`. The initial certificate is self-signed — use the Certs tab to issue a real one once DNS is propagating.
@@ -73,6 +73,7 @@ See [docs/deployment.md](docs/deployment.md) for the full guide, including troub
```bash
git clone <repo-url> && cd vacuum-wall
bash scripts/update-vendor.sh
python3 -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"
```
@@ -98,7 +99,7 @@ All `lib/` modules share `lib.common` utilities (`run`, `run_proc`, `load_json`,
.venv/bin/python -m pytest tests/ -v
```
Tests mock all subprocess calls (firewalld, acme.sh, WireGuard, nginx, dnsmasq) — no system services required. 192 tests across 5 test modules.
Tests mock all subprocess calls (firewalld, acme.sh, WireGuard, nginx, dnsmasq) — no system services required. 28 Python test files (pytest) + 9 JS test files (jsdom/node harness).
### Documentation MCP Server
@@ -114,17 +115,23 @@ Then run with Claude Code or Opencode to activate it. It automatically checks li
### Architecture
```
Client ──→ nginx (SSL + basic auth) ──→ Flask (127.0.0.1:9090)
Flask ──→ lib/*.py ──→ sudo <cmd> ──→ system service
Client ──→ nginx (TLS; basic auth on basic-authed proxy domains only) ──→ Flask (127.0.0.1:9090)
Flask ──→ daemon/client.py (Unix socket) ──→ vacuum-walld ──→ handlers ──→ sudo
```
| Blueprint | URL prefix | Backend module |
|---|---|---|
| `webui/api/firewall` | `/api/firewall/` | `lib.firewall` |
| `webui/api/dhcp` | `/api/dhcp/` | `lib.dnsmasq` |
| `webui/api/proxy` | `/api/proxy/` | `lib.nginx` |
| `webui/api/certs` | `/api/certs/` | `lib.acme` |
| `webui/api/wireguard` | `/api/wireguard/` | `lib.wireguard` |
Blueprints are thin proxies — privileged handlers live in `daemon/handlers/*.py`.
| Blueprint | URL prefix | Handler | lib module |
|---|---|---|---|
| `webui/api/firewall` | `/api/firewall/` | `daemon/handlers/firewall` | `lib.firewall` |
| `webui/api/dhcp` | `/api/dhcp/` | `daemon/handlers/dnsmasq` | `lib.dnsmasq` |
| `webui/api/proxy` | `/api/proxy/` | `daemon/handlers/nginx` | `lib.nginx` |
| `webui/api/certs` | `/api/certs/` | `daemon/handlers/acme` | `lib.acme` |
| `webui/api/wireguard` | `/api/wireguard/` | `daemon/handlers/wireguard` | `lib.wireguard` |
| `webui/api/network` | `/api/network/` | `daemon/handlers/network` | `lib.network` |
| `webui/api/logs` | `/api/logs/` | `daemon/handlers/logs` | — |
| `webui/api/status` | `/api/status/` | `daemon/handlers/status` | — |
| `webui/api/auth` | `/api/auth/` | `daemon/handlers/auth` | `lib.auth` / `lib.auth_users` |
See [docs/architecture.md](docs/architecture.md) for detailed request flow, zone model, and shared utility patterns.
@@ -138,3 +145,5 @@ See [docs/architecture.md](docs/architecture.md) for detailed request flow, zone
- [API Reference](docs/api.md) — REST API endpoints
- [Security Model](docs/security.md) — Privilege model and sudo whitelist
- [Configuration](docs/config.md) — Declarative config file formats and locations
- [State Model](docs/state-model.md) — Per-subsystem state schema and real-time push mechanics
- [Frontend (hoover)](docs/hoover.md) — Custom reactive SPA framework API reference
+9
View File
@@ -94,6 +94,15 @@ def _format_path(path: str, params: dict[str, Any] | None) -> str:
return path
def _replace(m: re.Match[str]) -> str:
"""Regex callback that replaces ``<key>`` segments with URL-encoded values.
Args:
m: Match object containing the parameter name.
Returns:
URL-encoded value from params dict, or original text if key
not found.
"""
key = m.group(1)
if key in params:
return urllib.parse.quote(str(params[key]), safe="")
+26
View File
@@ -0,0 +1,26 @@
"""State collectors for vacuum-walld.
Importing this package registers every collector with the ``lib.state``
store (registration side effect). Import it before the first
``populate()``/``poll()`` call.
"""
from daemon.collectors import (
acme,
dnsmasq,
firewall,
networkd,
nginx,
system,
wireguard,
)
__all__ = [
"acme",
"dnsmasq",
"firewall",
"networkd",
"nginx",
"system",
"wireguard",
]
+225
View File
@@ -0,0 +1,225 @@
"""ACME state collector."""
import logging
import os
from pathlib import Path
from stat import S_IRGRP
from typing import Any
from lib import schema
from lib.acme import get_acme_home
from lib.common import load_json
from lib.state import PROJECT_DIR, _now_iso, register_collector
logger = logging.getLogger(__name__)
_CA_NAME_MAP: dict[str, str] = {
"letsencrypt": "Let's Encrypt",
"zerossl": "ZeroSSL",
}
def _resolve_ca_name(ca_server: str) -> str:
"""Map a CA server identifier to its human-readable name.
Uses prefix matching sorted by longest prefix first to avoid
shorter prefixes winning (e.g. "letsencrypt" matching before
"letsencrypt.org").
Args:
ca_server: Raw CA server string from acme.sh config.
Returns:
Human-readable name, or unchanged string if no match.
"""
for prefix, name in sorted(
_CA_NAME_MAP.items(), key=lambda x: len(x[0]), reverse=True
):
if ca_server.startswith(prefix):
return name
return ca_server
def _parse_account_conf(acme_home: Path | None = None) -> dict[str, Any]:
"""Parse acme.sh account information and return account status dict.
Checks three sources in order:
1. Legacy ``.account.conf`` file (acme.sh v2.x format)
2. Declarative ``config/acme/config.json`` (saved by the registration
handler with ``email`` and ``ca`` fields)
Args:
acme_home: Optional override for ACME home directory. Falls back
to ``ACME_HOME`` env var or ``PROJECT_DIR/data/acme``.
Returns:
Dict with ``registered``, ``email``, ``ca``, and
``key_length`` keys. If no account is found, ``registered`` is
``False`` with empty / ``None`` values.
"""
if acme_home is None:
acme_home_env = os.environ.get("ACME_HOME", str(PROJECT_DIR / "data" / "acme"))
acme_home = Path(acme_home_env)
default = {
"registered": False,
"email": "",
"ca": "",
"key_length": None,
}
# 1. acme.sh account file. Modern acme.sh (v3.x) writes ``account.conf``;
# older v2.x wrote ``.account.conf``. Check both so the account card
# reflects the real acme.sh account rather than only the declarative
# fallback below.
account_path = None
for name in ("account.conf", ".account.conf"):
candidate = acme_home / name
if candidate.is_file():
account_path = candidate
break
if account_path is not None:
try:
text = account_path.read_text()
except OSError:
pass
else:
email = ""
ca_raw = ""
key_length = None
for line in text.splitlines():
if line.startswith("ACME_LEEMAIL="):
email = line.split("=", 1)[1].strip().strip("'\"")
elif line.startswith("ACME_MCA="):
ca_raw = line.split("=", 1)[1].strip().strip("'\"")
elif line.startswith("ACME_CERTKEYSIZE="):
raw_val = line.split("=", 1)[1].strip().strip("'\"")
key_length = int(raw_val) if raw_val.isdigit() else None
if email and ca_raw:
return {
"registered": True,
"email": email,
"ca": _resolve_ca_name(ca_raw),
"key_length": key_length,
}
# 2. Declarative config (saved by register_account / set_email handlers)
# Modern acme.sh (v3.x) stores account data in per-CA JSON files
# (ca/<server>/account.json) — we can't reliably parse those without
# walking the directory, so fall back to the declarative config
# which the handlers keep in sync.
# Derive project root from acme_home (acme_home is at <root>/data/acme).
try:
project_root = acme_home.parent.parent # data/acme → data → project root
acme_cfg = project_root / "config" / "acme" / "config.json"
data = load_json(acme_cfg)
email = (data.get("email") or "").strip()
ca_raw = (data.get("ca") or "").strip()
if email and ca_raw:
return {
"registered": True,
"email": email,
"ca": _resolve_ca_name(ca_raw),
"key_length": None,
}
except (OSError, ValueError):
pass
return default
def _get_acme_email() -> str:
"""Read the ACME ``acme.sh`` email from the account config file.
Falls back to the declarative ACME config (config/acme/config.json)
if acme.sh account has not been registered yet.
"""
from lib.acme import _read_acme_email
return _read_acme_email()
def _friendly_acme_error(exc: Exception) -> str:
"""Turn a collection exception into an actionable message.
The collector already self-heals by normalizing ACME_HOME permissions
first, so the one remaining permission case is when that normalize could
not run (e.g. the sudo step was denied). For that case surface a concrete
remediation instead of the raw acme.sh exit-2 text; otherwise return the
original message unchanged.
"""
text = str(exc)
if "account.conf" in text and "Permission denied" in text:
return (
f"{text} — account.conf is not readable by the daemon; repair it "
"with: sudo chown <daemon-user>:<group> <ACME_HOME>/account.conf "
"&& sudo chmod 0640 <ACME_HOME>/account.conf, then restart "
"vacuum-walld"
)
return text
def _acme_home_needs_normalize() -> bool:
"""Cheap no-sudo probe: has any ACME_HOME file lost its group-read bit?
acme.sh re-hardens its tree (``chmod 600``) on every run, so the daemon's
self-heal (``normalize_acme_home``) is only needed after a run by another
user (e.g. a manual run as the WebUI user) stripped group read. The probe
checks the group bit — not the daemon's own readability — because group
read is what the two-user model keeps for the WebUI user; a file the
daemon can read but the group cannot must still be healed.
"""
try:
for p in get_acme_home().rglob("*"):
if p.is_file() and not (p.stat().st_mode & S_IRGRP):
return True
except OSError:
return True
return False
def _collect_acme() -> schema.AcmeState:
"""Collect ACME certificate list and email.
Returns:
Dict containing certificate details and registered email.
"""
email = _get_acme_email()
# Non-fatal: a broken acme.sh (e.g. unreadable account.conf after an
# ownership flip) must not blank the whole dashboard via a cleared
# state store. Collect what we can and surface the failure in
# `status.error` so the poll diff still detects recovery.
cert_error: str | None = None
try:
# Self-heal ACME_HOME permissions before listing, but only when the
# probe detects a lost group-read bit — the steady-state poll then
# makes no sudo call. acme.sh dot-sources account.conf on startup; a
# prior run by another user (e.g. a manual run as the WebUI user) can
# leave it owner-only and make `--list` exit 2. The startup normalize
# only covers the first collection, so the poll must probe too or a
# mid-lifetime ownership flip would blank the cert list until the
# next issue/renew or daemon restart.
from daemon.handlers.acme import normalize_acme_home
from lib.acme import list_certs
if _acme_home_needs_normalize():
normalize_acme_home()
certs = list_certs()
except Exception as exc:
logger.warning("ACME state collection failed", exc_info=True)
certs = []
cert_error = _friendly_acme_error(exc)
account = _parse_account_conf()
return {
"certs": certs,
"email": email,
"account": account,
"status": {"error": cert_error},
"timestamp": _now_iso(),
}
register_collector("acme", _collect_acme)
+85
View File
@@ -0,0 +1,85 @@
"""DNSMasq state collector."""
from copy import deepcopy
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from lib import schema
from lib.common import compute_pending, run_proc, strip_apply_meta
from lib.dnsmasq import DEFAULT_CFG, get_config
from lib.state import _now_iso, register_collector
def _collect_dnsmasq() -> schema.DnsmasqState:
"""Collect dnsmasq status, config, and leases.
Returns:
Dict containing config, service status, leases, and timestamp.
"""
DNSMASQ_CONF = "/etc/dnsmasq.d/vacuum-wall.conf"
LEASE_FILE = "/var/lib/misc/dnsmasq.leases"
# Load config (lib defaults; fall back to them when the file is broken)
try:
cfg = get_config()
except Exception:
cfg = deepcopy(DEFAULT_CFG)
# Service status
service_active = False
try:
proc = run_proc(["systemctl", "is-active", "dnsmasq"], sudo=True)
service_active = proc.stdout.strip() == "active"
except Exception:
pass
# Leases
leases: list[dict[str, Any]] = []
try:
result = run_proc(["cat", LEASE_FILE], sudo=True, check=True)
for line in result.stdout.splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
parts = line.split()
if len(parts) < 3:
continue
try:
ts = datetime.fromtimestamp(int(parts[0]), tz=UTC)
except (ValueError, OSError):
ts = None
leases.append(
{
"expires": ts.isoformat() if ts else "",
"mac": parts[1],
"ip": parts[2],
"hostname": parts[3] if len(parts) > 3 else "",
"interface": parts[4] if len(parts) > 4 else "",
}
)
except Exception:
pass
# Check config file on disk
conf_exists = Path(DNSMASQ_CONF).is_file()
pending_changes, pending_diff = compute_pending(cfg)
safe_cfg = strip_apply_meta(cfg)
return {
"config": safe_cfg,
"status": {
"service_active": service_active,
"config_file_exists": conf_exists,
"active_leases": len(leases),
"pending_changes": pending_changes,
"pending_diff": pending_diff,
},
"leases": leases,
"timestamp": _now_iso(),
}
register_collector("dnsmasq", _collect_dnsmasq)
# dnsmasq has no volatile fields — leases change slowly enough to treat as structural
+175
View File
@@ -0,0 +1,175 @@
"""Firewall state collector (read-only sudo queries)."""
import contextlib
from typing import Any
from lib import schema
from lib.common import load_json, run, strip_apply_meta
from lib.firewall import (
_parse_active_zones,
_parse_all_zones_output,
get_service_descriptions,
)
from lib.firewall import (
config_pending as _config_pending,
)
from lib.network import get_config as _network_get_config
from lib.state import (
PROJECT_DIR,
_now_iso,
register_collector,
register_volatile,
)
def _fp_to_str(fp: dict[str, Any]) -> str:
"""Convert a port-forward dict to a compact string representation.
Args:
fp: Port-forward entry containing port and proto keys.
Returns:
Comma-separated string of key=value pairs (e.g. ``port=443,proto=tcp``).
"""
parts = [f"port={fp['port']}", f"proto={fp['proto']}"]
if "toaddr" in fp:
parts.append(f"toaddr={fp['toaddr']}")
if "toport" in fp:
parts.append(f"toport={fp['toport']}")
return "/".join(parts)
def _collect_firewall() -> schema.FirewallState:
"""Return the complete current state of firewalld.
Returns:
Dict containing firewall zones, interfaces, rules, config, and
pending changes.
"""
active_raw = run(["firewall-cmd", "--get-active-zones"], sudo=True)
active = _parse_active_zones(active_raw)
default_zone = run(["firewall-cmd", "--get-default-zone"], sudo=True).strip()
services = run(["firewall-cmd", "--get-services"], sudo=True).split() or []
link_out = run(["ip", "-o", "link", "show"], sudo=True)
addr_out = run(["ip", "-o", "addr", "show"], sudo=True)
iface_map: dict[str, dict[str, Any]] = {}
for line in link_out.splitlines():
if not line:
continue
parts = line.split()
if len(parts) < 2:
continue
raw_name = parts[1].rstrip(":").split("@")[0]
iface_state = "UNKNOWN"
mtu = None
mac = None
for i, p in enumerate(parts):
if p == "state" and i + 1 < len(parts):
iface_state = parts[i + 1]
if p == "mtu" and i + 1 < len(parts):
mtu = int(parts[i + 1])
if p.startswith("link/ether") and i + 1 < len(parts):
mac = parts[i + 1]
iface_map[raw_name] = {
"name": raw_name,
"mac": mac,
"state": iface_state,
"mtu": mtu,
"ips": [],
"ipv6": [],
"zone": None,
}
for line in addr_out.splitlines():
if not line:
continue
parts = line.split()
if len(parts) < 4:
continue
addr_name = parts[1].split("@")[0]
addr_key = "ipv6" if parts[2] == "inet6" else "ips"
for entry in iface_map.values():
if entry["name"] == addr_name:
entry[addr_key].append(parts[3])
break
for zone_name, ifaces in active.items():
for raw_if in ifaces:
for entry in iface_map.values():
if entry["name"] == raw_if:
entry["zone"] = zone_name
break
ifaces = list(iface_map.values())
# Collect all zones in a single call (replaces per-zone loop)
zones: dict[str, dict[str, Any]] = {}
try:
all_zones_raw = run(["firewall-cmd", "--list-all-zones"], sudo=True)
zones = _parse_all_zones_output(all_zones_raw)
except Exception:
pass
# Load config (strip apply bookkeeping keys, as the other collectors do)
fw_config_path = PROJECT_DIR / "config" / "firewall" / "config.json"
config_data = {}
if fw_config_path.exists():
with contextlib.suppress(Exception):
config_data = strip_apply_meta(load_json(fw_config_path))
# Pending changes
full_state = {
"active_zones": active,
"default_zone": default_zone,
"interfaces": ifaces,
"available_services": services,
"zones": zones,
"rich_rules": {n: z.get("rich-rules", []) for n, z in zones.items()},
"timestamp": _now_iso(),
}
pending = {}
with contextlib.suppress(Exception):
pending = _config_pending(full_state)
net_cfg: dict[str, Any] = {}
with contextlib.suppress(Exception):
net_cfg = _network_get_config()
covered: set[str] = set()
for zone_ifaces in active.values():
covered.update(zone_ifaces)
for zone in zones.values():
covered.update(zone.get("interfaces", []))
uncovered_interfaces = [
name
for name in net_cfg.get("interfaces", {})
if name != "lo" and not name.startswith("wg") and name not in covered
]
return {
"active_zones": active,
"default_zone": default_zone,
"interfaces": ifaces,
"available_services": services,
# Parsed from the firewalld service XML definitions; cached per
# process so the 30s poll does not re-read the files.
"service_descriptions": get_service_descriptions(),
"uncovered_interfaces": uncovered_interfaces,
"zones": zones,
"rich_rules": {n: z.get("rich-rules", []) for n, z in zones.items()},
"config": config_data,
"pending": pending,
"timestamp": _now_iso(),
}
register_collector("firewall", _collect_firewall)
register_volatile(
"firewall",
frozenset(
{
"interfaces[].ips",
"interfaces[].ipv6",
}
),
)
+67
View File
@@ -0,0 +1,67 @@
"""Networkd state collector."""
from typing import Any
from lib import schema
from lib.common import compute_pending, run, strip_apply_meta
from lib.network import get_config, parse_networkctl_status
from lib.state import _now_iso, register_collector, register_volatile
def _collect_networkd() -> schema.NetworkdState:
"""Collect networkd interface state from networkctl.
Returns:
Dict with interface runtime state parsed from networkctl output,
config, and pending changes status.
"""
# Load config
try:
net_cfg = get_config()
except Exception:
net_cfg = {}
pending_changes, net_pending_diff = compute_pending(net_cfg)
result: dict[str, dict[str, Any]] = {}
safe_net_cfg = strip_apply_meta(net_cfg)
net_status: dict[str, Any] = {
"pending_changes": pending_changes,
"pending_diff": net_pending_diff,
}
try:
raw = run(["networkctl", "status", "--json=short", "--all"], sudo=True)
result = parse_networkctl_status(raw)
if not result:
return {
"interfaces": {},
"config": safe_net_cfg,
"status": net_status,
"timestamp": _now_iso(),
}
except Exception:
return {
"interfaces": {},
"config": safe_net_cfg,
"status": net_status,
"timestamp": _now_iso(),
}
return {
"interfaces": result,
"config": safe_net_cfg,
"status": net_status,
"timestamp": _now_iso(),
}
register_collector("networkd", _collect_networkd)
register_volatile(
"networkd",
frozenset(
{
"interfaces[].addresses",
}
),
)
+66
View File
@@ -0,0 +1,66 @@
"""Nginx state collector."""
from copy import deepcopy
from typing import Any
from lib import schema
from lib.common import compute_pending, strip_apply_meta
from lib.nginx import DEFAULT_CONFIG, SITES_DIR, _resolve_paths, get_config
from lib.state import _now_iso, register_collector
def _collect_nginx() -> schema.NginxState:
"""Collect nginx config and domains list.
Returns:
Dict containing config, domains, and timestamp.
"""
# Load config via lib.nginx — a pure read that applies the legacy-format
# migration in memory (the one-shot on-disk migration runs at daemon
# startup, see lib.bootstrap).
try:
cfg = get_config()
except Exception:
cfg = deepcopy(DEFAULT_CONFIG)
# Build flattened domains list (one entry per path)
backends = cfg.get("backends", {})
domains: list[dict[str, Any]] = []
for name, dom in cfg.get("domains", {}).items():
if "backend" not in dom:
continue
site = SITES_DIR / f"{name}.conf"
paths = _resolve_paths(dom, backends)
if not paths:
continue
for ppath, pcfg in paths.items():
entry: dict[str, Any] = {
"domain": name,
"path": ppath,
"backend": pcfg.get("backend", {}),
"online": site.exists() if SITES_DIR.exists() else False,
"force_ssl": dom.get("force_ssl", True),
"backend_name": dom["backend"],
"cert": dom.get("cert"),
}
if pcfg.get("is_management"):
entry["is_management"] = True
if pcfg.get("is_websocket"):
entry["is_websocket"] = True
domains.append(entry)
pending_changes, nginx_pending_diff = compute_pending(cfg)
safe_cfg = strip_apply_meta(cfg)
return {
"config": safe_cfg,
"domains": domains,
"status": {
"pending_changes": pending_changes,
"pending_diff": nginx_pending_diff,
},
"timestamp": _now_iso(),
}
register_collector("nginx", _collect_nginx)
+125
View File
@@ -0,0 +1,125 @@
"""System metrics collector."""
from pathlib import Path
from typing import Any
from lib import schema
from lib.state import _now_iso, register_collector, register_volatile
def _parse_meminfo() -> dict[str, Any]:
"""Read /proc/meminfo and return dict with key memory stats in bytes."""
info: dict[str, int] = {}
try:
for line in Path("/proc/meminfo").read_text().splitlines():
if ":" not in line:
continue
key, value = line.split(":", 1)
key = key.strip()
parts = value.strip().split()
val = int(parts[0])
# Convert kB to bytes
if parts and parts[-1] == "kB":
val *= 1024
info[key] = val
except (OSError, ValueError):
return {}
return info
def _collect_system() -> schema.SystemState:
"""Collect system-wide metrics: CPU load, memory, network traffic.
Reads from /proc and /sys — no subprocess needed.
Returns:
Dict with load (1/5/15 min), memory usage, and per-interface traffic.
"""
# CPU load
loads = []
try:
parts = Path("/proc/loadavg").read_text().split()
loads = [float(x) for x in parts[:3]]
except (OSError, ValueError):
loads = [0.0, 0.0, 0.0]
# Memory
meminfo_raw = _parse_meminfo()
mem_total = meminfo_raw.get("MemTotal", 0)
mem_free = meminfo_raw.get("MemFree", 0)
mem_available = meminfo_raw.get("MemAvailable", mem_free)
mem_buffers = meminfo_raw.get("Buffers", 0)
mem_cached = meminfo_raw.get("Cached", 0)
mem_used = mem_total - mem_free - mem_buffers - mem_cached
if mem_used < 0:
mem_used = mem_total - mem_available
# Swap
swap_total = meminfo_raw.get("SwapTotal", 0)
swap_free = meminfo_raw.get("SwapFree", 0)
swap_used = swap_total - swap_free
# Network traffic from /sys/class/net/<iface>/statistics/
traffic: dict[str, dict[str, int]] = {}
try:
net_root = Path("/sys/class/net")
if net_root.is_dir():
for iface_dir in net_root.iterdir():
stats_dir = iface_dir / "statistics"
if not stats_dir.is_dir():
continue
iface_name = iface_dir.name
rx_bytes = 0
tx_bytes = 0
rx_packets = 0
tx_packets = 0
try:
rx_bytes = int((stats_dir / "rx_bytes").read_text().strip())
tx_bytes = int((stats_dir / "tx_bytes").read_text().strip())
rx_packets = int((stats_dir / "rx_packets").read_text().strip())
tx_packets = int((stats_dir / "tx_packets").read_text().strip())
except (OSError, ValueError):
continue
traffic[iface_name] = {
"rx_bytes": rx_bytes,
"tx_bytes": tx_bytes,
"rx_packets": rx_packets,
"tx_packets": tx_packets,
}
except OSError:
pass
return {
"load": {
"load1": loads[0],
"load5": loads[1],
"load15": loads[2],
},
"memory": {
"total": mem_total,
"available": mem_available,
"used": mem_used,
"used_pct": round(mem_used / mem_total * 100, 1) if mem_total > 0 else 0,
},
"swap": {
"total": swap_total,
"used": swap_used,
"used_pct": round(swap_used / swap_total * 100, 1) if swap_total > 0 else 0,
},
"traffic": traffic,
"timestamp": _now_iso(),
}
register_collector("system", _collect_system)
register_volatile(
"system",
frozenset(
{
"load",
"memory",
"swap",
"traffic",
}
),
)
+125
View File
@@ -0,0 +1,125 @@
"""WireGuard state collector."""
from copy import deepcopy
from typing import Any
from lib import schema
from lib.common import compute_pending, run_proc, strip_apply_meta
from lib.state import _now_iso, register_collector, register_volatile
from lib.wireguard import DEFAULT_CONFIG, get_config, parse_wg_show_output
def _collect_wireguard() -> schema.WgState:
"""Collect WireGuard config, per-class status, and peers.
Returns:
Dict containing interface config, per-class runtime status,
combined peers, and overall tunnel status.
"""
# Load config via lib.wireguard defaults (which include the built-in
# full/internet access classes).
try:
cfg = get_config()
except Exception:
cfg = deepcopy(DEFAULT_CONFIG)
pending_changes, pending_diff = compute_pending(cfg)
# Safe config (strip private keys from interface and access classes)
safe = strip_apply_meta(cfg)
if "interface" in safe:
safe["interface"] = dict(safe["interface"])
safe["interface"].pop("private_key", None)
if "access_classes" in safe:
safe["access_classes"] = {}
for ck, cv in cfg.get("access_classes", {}).items():
if isinstance(cv, dict):
entry = dict(cv)
entry.pop("private_key", None)
safe["access_classes"][ck] = entry
# Peers list (safe)
peers: list[dict[str, Any]] = []
for name, info in cfg.get("peers", {}).items():
entry = dict(info)
entry["name"] = name
entry.pop("private_key", None)
peers.append(entry)
# Runtime status — per-class interfaces
status: dict[str, Any] = {
"up": False,
"interface": {},
"peers": [],
"classes": {},
}
classes = cfg.get("access_classes", {})
any_up = False
for class_key in classes:
class_cfg = classes.get(class_key)
if not isinstance(class_cfg, dict):
continue
ifname = f"wg-{class_key}"
try:
res = run_proc(["wg", "show", ifname], sudo=True, check=False)
if res.returncode != 0:
status["classes"][class_key] = {
"up": False,
"interface": {},
"peers": [],
}
continue
parsed = parse_wg_show_output(res.stdout.strip())
status["classes"][class_key] = {
"up": parsed["up"],
"interface": parsed.get("interface", {}),
"peers": parsed.get("peers", []),
}
if parsed["up"]:
any_up = True
except Exception:
status["classes"][class_key] = {"up": False, "interface": {}, "peers": []}
# Also collect legacy single-interface status
try:
ifname = cfg["interface"].get("name", "wg0")
res = run_proc(["wg", "show", ifname], sudo=True, check=False)
if res.returncode == 0:
parsed = parse_wg_show_output(res.stdout.strip())
status["up"] = True
status["interface"] = parsed.get("interface", {})
status["peers"] = parsed.get("peers", [])
any_up = True
except Exception:
pass
if any_up:
status["up"] = True
status["pending_changes"] = pending_changes
# Drop any private-key paths so the pending summary never exposes
# key material.
status["pending_diff"] = [d for d in pending_diff if "private_key" not in d["path"]]
return {
"config": safe,
"status": status,
"peers": peers,
"timestamp": _now_iso(),
}
register_collector("wireguard", _collect_wireguard)
register_volatile(
"wireguard",
frozenset(
{
"status.peers[].transfer_received",
"status.peers[].transfer_sent",
"status.peers[].latest_handshake",
"status.classes[].peers[].transfer_received",
"status.classes[].peers[].transfer_sent",
"status.classes[].peers[].latest_handshake",
}
),
)
+203 -39
View File
@@ -26,6 +26,7 @@ from daemon.iface import (
GET_ACME_ISSUE_STATUS,
GET_ACME_LIST,
GET_ACME_PATHS,
GET_ACME_RENEW_STATUS,
POST_ACME_ACCOUNT_REGISTER,
POST_ACME_EMAIL,
POST_ACME_ISSUE,
@@ -53,9 +54,57 @@ _ACME_ENVIRON = {
_WEBROOT = PROJECT_DIR / "data" / "acme" / "www"
def normalize_acme_home() -> None:
"""Restore group access on the ACME home files around acme.sh runs.
acme.sh hardens its tree on every run (``chmod 700`` on the config
home, ``chmod 600`` on keys and confs, owned by the running user).
The daemon reopens group read/write via the sudoers whitelist so
the shared two-user model keeps the tree readable. Run BEFORE an
acme.sh invocation too: acme.sh dot-sources ``account.conf`` on
startup, so a tree left owner-only by another user's run (e.g. a
manual debug run as the WebUI user) would make every daemon acme.sh
call exit 2 — normalizing first is the only self-heal path, since a
post-run normalize is unreachable while acme.sh cannot start.
Files only: the directories in the tree are setgid (2775, group rwx
already), and chmodding a setgid directory issues fchmodat with the
S_ISGID bit set, which the unit's ``RestrictSUIDSGID=yes`` seccomp
filter rejects with EPERM even for root.
"""
files = [str(p) for p in _ACME_HOME.rglob("*") if p.is_file()]
if not files:
return
result = lib_common.run_proc(
["chmod", "g+rwX", *files],
sudo=True,
check=False,
timeout=10,
)
if result.returncode != 0:
logger.warning(
"Could not normalize ACME_HOME permissions: %s",
result.stderr.strip() or f"exit code {result.returncode}",
)
def _run_acme_preflight(args: list[str]) -> str:
"""Normalize ACME home permissions, then run acme.sh with *args*.
Single choke point for every daemon acme.sh invocation: the
preflight normalize makes the run succeed even if a prior run by
another user left the tree owner-only.
"""
normalize_acme_home()
return _run_acme(args)
# In-memory store for active issuance requests.
_ISSUANCES: dict[str, "IssueRequest"] = {}
_ISSUANCE_TTL = 300 # seconds to keep completed requests
_ISSUANCE_TASKS: dict[str, asyncio.Task] = {}
def _find_issuance(domain: str) -> "IssueRequest | None":
@@ -172,6 +221,21 @@ def _clean_expired_issuances() -> None:
del _ISSUANCES[rid]
def _fail_op(req: IssueRequest, exc: Exception) -> None:
"""Record the error on the first running step and mark the request failed."""
for step in req.steps:
if step.status == "running":
step.status = "error"
step.message = str(exc)
break
else:
req.steps.append(
IssueStep(name="error", label="Error", status="error", message=str(exc))
)
req.status = "failed"
req.expires_at = datetime.now(UTC).timestamp() + _ISSUANCE_TTL
# ---------------------------------------------------------------------------
# Validation helpers
@@ -528,7 +592,7 @@ def _check_acme_account() -> tuple[bool, str]:
except (FileNotFoundError, subprocess.TimeoutExpired):
pass
from lib.state import _parse_account_conf
from daemon.collectors.acme import _parse_account_conf
info = _parse_account_conf(_ACME_HOME)
if info.get("registered"):
@@ -543,11 +607,11 @@ def _check_acme_account() -> tuple[bool, str]:
def _check_account_registered() -> tuple[bool, str]:
"""Blocking check: verify an ACME account is registered.
Delegates to ``lib.state._parse_account_conf()`` which checks both
Delegates to ``daemon.collectors.acme._parse_account_conf()`` which checks both
the legacy .account.conf and the declarative config/acme/config.json
used by modern acme.sh (v3.x).
"""
from lib.state import _parse_account_conf
from daemon.collectors.acme import _parse_account_conf
info = _parse_account_conf(_ACME_HOME)
if info.get("registered"):
@@ -558,10 +622,10 @@ def _check_account_registered() -> tuple[bool, str]:
def _get_account_info() -> dict[str, Any]:
"""Read and return the ACME account info dict.
Delegates to ``lib.state._parse_account_conf()`` for a single
Delegates to ``daemon.collectors.acme._parse_account_conf()`` for a single
source of truth.
"""
from lib.state import _parse_account_conf
from daemon.collectors.acme import _parse_account_conf
return _parse_account_conf(_ACME_HOME)
@@ -743,8 +807,11 @@ async def issue_cert(_request: Any, body: dict[str, Any] | None) -> dict[str, An
"status": "existing",
}
# Check if cert already exists — call acme.sh directly, not via state
# Check if cert already exists — call acme.sh directly, not via state.
# Normalize ACME_HOME first (same reason as the preflight): a prior run by
# another user can leave account.conf owner-only and make `--list` exit 2.
try:
normalize_acme_home()
certs = lib.acme.list_certs()
except RuntimeError as exc:
raise RuntimeError(f"Cannot check existing certificates: {exc}") from exc
@@ -783,7 +850,8 @@ async def issue_cert(_request: Any, body: dict[str, Any] | None) -> dict[str, An
_ISSUANCES[request_id] = req
# Spawn background task
_task = asyncio.create_task(_run_issue(req)) # noqa: RUF006 — task runs to completion on its own
_task = asyncio.create_task(_run_issue(req))
_ISSUANCE_TASKS[request_id] = _task
return {"request_id": request_id, "domain": domain}
@@ -818,13 +886,19 @@ async def _run_issue(req: IssueRequest) -> None:
if account_email:
args.extend(["-m", account_email])
args.append("--force")
output = _run_acme(args)
# acme.sh is a blocking subprocess — run it off the event loop so
# polling, WS broadcasts, and other requests keep responding.
output = await asyncio.to_thread(_run_acme_preflight, args)
normalize_acme_home()
req.steps[0].status = "done"
req.steps[0].message = output.strip()[:200]
# Step 2: deploy
req.steps[1].status = "running"
_run_acme(["--deploy", "-d", req.domain, "--deploy-hook", _DEPLOY_HOOK])
await asyncio.to_thread(
_run_acme_preflight,
["--deploy", "-d", req.domain, "--deploy-hook", _DEPLOY_HOOK],
)
req.steps[1].status = "done"
req.steps[1].message = "Deploy hook registered"
@@ -841,44 +915,134 @@ async def _run_issue(req: IssueRequest) -> None:
)
except Exception as exc:
# Mark current running step as error, overall as failed
for step in req.steps:
if step.status == "running":
step.status = "error"
step.message = str(exc)
break
else:
req.steps.append(
IssueStep(name="error", label="Error", status="error", message=str(exc))
)
req.status = "failed"
req.expires_at = datetime.now(UTC).timestamp() + _ISSUANCE_TTL
_fail_op(req, exc)
logger.error("Cert issuance for %s failed: %s", req.domain, exc)
finally:
_ISSUANCE_TASKS.pop(req.request_id, None)
@registry.register(POST_ACME_RENEW)
def renew_cert(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /acme/renew — renew a certificate for the given domain.
"""POST /acme/renew — start a certificate renewal request (async).
Deduplicates in-progress requests per domain. Spawns a background task
for the actual renewal. When ``force`` is not set, acme.sh skips the
renewal if the certificate's renewal window has not been reached yet
(the request then completes with status "skipped").
Args:
force: Force renewal regardless of expiry.
Returns:
A dictionary containing the renewal request id (and "existing"
status when a renewal for the domain is already running).
Raises:
ValueError: When domain is missing.
"""
if not body:
raise ValueError("Request body required")
domain = body.get("domain", "").strip()
domain = (body.get("domain") or "").strip()
if not domain:
raise ValueError("'domain' is required")
force = body.get("force", False)
args: list[str] = ["--renew", "-d", domain]
if force:
args.append("--force")
output = _run_acme(args)
_run_acme(["--deploy", "-d", domain, "--deploy-hook", _DEPLOY_HOOK])
logger.info("Certificate for %s renewed", domain)
refresh_state(["acme"])
return {"domain": domain, "output": output.strip()}
force = bool(body.get("force", False))
_clean_expired_issuances()
# Dedup: if domain already has an active request, return it
existing = _find_issuance(domain)
if existing:
return {
"request_id": existing.request_id,
"domain": domain,
"status": "existing",
}
request_id = uuid4().hex[:12]
steps = [
IssueStep(name="renew", label="Renewing certificate"),
IssueStep(name="deploy", label="Registering deploy hook"),
IssueStep(name="refresh", label="Refreshing certificate state"),
]
req = IssueRequest(request_id=request_id, domain=domain, steps=steps)
_ISSUANCES[request_id] = req
# Spawn background task
_task = asyncio.create_task(_run_renew(req, force))
_ISSUANCE_TASKS[request_id] = _task
return {"request_id": request_id, "domain": domain}
@registry.register(GET_ACME_RENEW_STATUS)
def get_renew_status(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""GET /acme/renew/status — poll status of a renewal request.
Raises:
ValueError: When id is missing.
NotFoundError: When request_id is unknown.
"""
request_id = (body or {}).get("id", "").strip()
if not request_id:
raise ValueError("'id' is required")
req = _ISSUANCES.get(request_id)
if not req:
raise NotFoundError(f"Renewal request {request_id} not found")
return req.to_dict()
async def _run_renew(req: IssueRequest, force: bool) -> None:
"""Background task: renew the certificate, register the deploy hook, refresh state."""
try:
# Step 1: renew (acme.sh skips when the cert's renewal window has not
# been reached unless force is set)
req.steps[0].status = "running"
args: list[str] = ["--renew", "-d", req.domain]
if force:
args.append("--force")
output = await asyncio.to_thread(_run_acme_preflight, args)
# acme.sh hardens its tree even when it skips — normalize first.
normalize_acme_home()
if "Skipping." in output:
req.steps[0].status = "done"
req.steps[0].message = "Renewal not yet due — skipped"
req.status = "skipped"
req.expires_at = datetime.now(UTC).timestamp() + _ISSUANCE_TTL
logger.info(
"Renewal for %s skipped (request %s)", req.domain, req.request_id
)
return
req.steps[0].status = "done"
req.steps[0].message = output.strip()[:200]
# Step 2: deploy
req.steps[1].status = "running"
await asyncio.to_thread(
_run_acme_preflight,
["--deploy", "-d", req.domain, "--deploy-hook", _DEPLOY_HOOK],
)
req.steps[1].status = "done"
req.steps[1].message = "Deploy hook registered"
# Step 3: refresh state
req.steps[2].status = "running"
refresh_state(["acme"])
req.steps[2].status = "done"
req.steps[2].message = "State refreshed"
req.status = "completed"
req.expires_at = datetime.now(UTC).timestamp() + _ISSUANCE_TTL
logger.info(
"Certificate for %s renewed (request %s)", req.domain, req.request_id
)
except Exception as exc:
_fail_op(req, exc)
logger.error("Renewal for %s failed: %s", req.domain, exc)
finally:
_ISSUANCE_TASKS.pop(req.request_id, None)
@registry.register(DELETE_ACME_REMOVE)
@@ -893,7 +1057,7 @@ def remove_cert(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
domain = body.get("domain", "").strip()
if not domain:
raise ValueError("'domain' is required")
_run_acme(["--remove", "-d", domain])
_run_acme_preflight(["--remove", "-d", domain])
logger.info("Certificate for %s removed", domain)
refresh_state(["acme"])
return {"domain": domain}
@@ -911,7 +1075,7 @@ def set_email(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
email = body.get("email", "").strip()
if not email:
raise ValueError("'email' is required")
_run_acme(["--register-account", "-m", email])
_run_acme_preflight(["--register-account", "-m", email])
# Persist to declarative ACME config
acme_cfg = PROJECT_DIR / "config" / "acme" / "config.json"
acme_cfg.parent.mkdir(parents=True, exist_ok=True)
@@ -976,10 +1140,10 @@ def generate_self_signed(_request: Any, body: dict[str, Any] | None) -> dict[str
raise ValueError("'domain' is required")
days = body.get("days", 365)
cert_dir = _ACME_HOME / domain
cert_dir.mkdir(parents=True, exist_ok=True)
cert_file = cert_dir / "fullchain.cer"
key_file = cert_dir / f"{domain}.key"
certs_dir = PROJECT_DIR / "data" / "certs"
certs_dir.mkdir(parents=True, exist_ok=True)
cert_file = certs_dir / f"{domain}.crt"
key_file = certs_dir / f"{domain}.key"
if cert_file.is_file() and key_file.is_file():
logger.info("Self-signed cert for %s already exists, skipping", domain)
@@ -1049,7 +1213,7 @@ def register_account(_request: Any, body: dict[str, Any] | None) -> dict[str, An
raise ValueError("Invalid email format")
server = (body.get("server") or "letsencrypt").strip()
_run_acme(["--register-account", "-m", email, "--server", server])
_run_acme_preflight(["--register-account", "-m", email, "--server", server])
acme_cfg = PROJECT_DIR / "config" / "acme" / "config.json"
acme_cfg.parent.mkdir(parents=True, exist_ok=True)
@@ -1069,7 +1233,7 @@ def register_account(_request: Any, body: dict[str, Any] | None) -> dict[str, An
def deactivate_account(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""DELETE /acme/account/deactivate — deactivate the ACME account."""
try:
_run_acme(["--deactivate-account"])
_run_acme_preflight(["--deactivate-account"])
except RuntimeError as exc:
logger.warning("acme.sh deactivate failed: %s", exc)
acme_cfg = PROJECT_DIR / "config" / "acme" / "config.json"
+593
View File
@@ -0,0 +1,593 @@
"""Authentication daemon handlers.
Handles login, logout, token refresh, session management, password change,
user CRUD, and WebAuthn operations.
"""
from __future__ import annotations
import logging
from typing import Any
from daemon.iface import (
DELETE_AUTH_USER,
DELETE_AUTH_WEBAUTHN_CREDENTIAL,
GET_AUTH_SESSION,
GET_AUTH_USERS,
GET_AUTH_WEBAUTHN_CAPABLE,
GET_AUTH_WEBAUTHN_CREDENTIAL_COUNTS,
GET_AUTH_WEBAUTHN_CREDENTIALS,
POST_AUTH_LOGIN,
POST_AUTH_LOGOUT,
POST_AUTH_PASSWORD,
POST_AUTH_REFRESH,
POST_AUTH_USER_CREATE,
POST_AUTH_USER_UPDATE,
POST_AUTH_WEBAUTHN_AUTHENTICATE_BEGIN,
POST_AUTH_WEBAUTHN_AUTHENTICATE_FINISH,
POST_AUTH_WEBAUTHN_REGISTER_BEGIN,
POST_AUTH_WEBAUTHN_REGISTER_FINISH,
)
from daemon.server import ConflictError, NotFoundError, registry
from lib.auth import (
blacklist_active_refresh_token,
blacklist_token,
check_login_rate,
check_webauthn_rate,
generate_tokens,
get_access_ttl,
record_login_failure,
record_login_success,
record_webauthn_failure,
record_webauthn_success,
validate_token,
)
from lib.auth_users import (
create_user,
delete_user,
get_user,
list_users,
update_password,
update_permissions,
verify_user_password,
)
from lib.webauthn import (
create_authentication_options,
create_registration_options,
get_all_credential_counts,
get_management_domains,
get_rp_name,
list_credentials,
remove_credential,
verify_authentication,
verify_registration,
)
from lib.webauthn import (
is_enabled as webauthn_is_enabled,
)
logger = logging.getLogger(__name__)
@registry.register(POST_AUTH_LOGIN)
def auth_login(_request: Any, body: Any) -> dict[str, Any]:
"""Handle user login.
Args:
_request: Unused.
body: Dict with ``username`` and ``password``.
Returns:
Dict with ``tokens``, ``user``, and ``permissions``.
Raises:
ValueError: If credentials are invalid.
"""
if not body or not isinstance(body, dict):
raise ValueError("Request body is required")
username = body.get("username")
password = body.get("password")
if not username or not password:
raise ValueError("username and password are required")
client_ip = body.get("client_ip")
if not check_login_rate(username, client_ip):
raise ValueError("Too many login attempts. Please try again later.")
user = verify_user_password(username, password)
if user is None:
record_login_failure(username, client_ip)
raise ValueError("Invalid credentials")
record_login_success(username, client_ip)
permissions = user["permissions"]
tokens = generate_tokens(username, permissions)
return {
"tokens": tokens,
"access_ttl": get_access_ttl(),
"user": {
"id": user["id"],
"username": user["username"],
},
"permissions": permissions,
}
@registry.register(POST_AUTH_LOGOUT)
def auth_logout(request: Any, body: Any) -> dict[str, Any]:
"""Handle user logout by blacklisting the access and refresh tokens.
Args:
request: The aiohttp request.
body: Dict with ``jti``, ``username`` from Flask user context, and
``refresh_token`` from the client.
Returns:
Success response.
"""
if not body:
raise ValueError("Request body is required")
jti = body.get("jti")
if jti:
blacklist_token(jti)
username = body.get("username")
if username:
blacklist_active_refresh_token(username)
return {}
@registry.register(POST_AUTH_REFRESH)
def auth_refresh(_request: Any, body: Any) -> dict[str, Any]:
"""Handle token refresh.
Validates the refresh token, blacklists it, and issues a new access token.
Requires refresh_token and session_id in the request body for session binding.
Args:
_request: Unused.
body: Dict with ``refresh_token`` and ``session_id``.
Returns:
Dict with new access token, refresh token, user, and permissions.
Raises:
ValueError: If refresh token is invalid.
"""
if not body or not isinstance(body, dict):
raise ValueError("Request body is required")
refresh_token = body.get("refresh_token")
if not refresh_token:
raise ValueError("refresh_token is required")
session_id = body.get("session_id")
if not session_id:
raise ValueError("session_id is required")
payload = validate_token(
refresh_token, token_type="refresh", session_id=session_id, require_session=True
)
if payload is None:
raise ValueError("Invalid or expired refresh token")
username = payload["sub"]
user = get_user(username)
if user is None:
raise ValueError("User not found")
# Persist new tokens first, then invalidate the old ones.
# This prevents data loss if generate_tokens fails mid-way:
# the old refresh token remains valid and the user is not locked out.
permissions = user["permissions"]
tokens = generate_tokens(username, permissions)
jti = payload.get("jti")
if jti:
blacklist_token(jti, token_type="refresh")
return {
"tokens": tokens,
"access_ttl": get_access_ttl(),
"user": {
"id": user["id"],
"username": user["username"],
},
"permissions": permissions,
}
@registry.register(GET_AUTH_SESSION)
def auth_session(request: Any, body: Any) -> dict[str, Any]:
"""Return current user session info.
Args:
request: The aiohttp request.
body: Dict with ``username`` from Flask user context.
Returns:
Dict with user info and permissions.
"""
username = body.get("username") if body else None
if username is None:
raise ValueError("No active session")
user = get_user(username)
if user is None:
raise ValueError("User not found")
return {
"user": {
"id": user["id"],
"username": user["username"],
},
"permissions": user["permissions"],
}
@registry.register(POST_AUTH_PASSWORD)
def auth_change_password(_request: Any, body: Any) -> dict[str, Any]:
"""Change user password.
Args:
_request: Unused.
body: Dict with ``username``, ``oldPassword``, ``newPassword``.
Returns:
Success response.
Raises:
ValueError: If password change fails.
"""
if not body or not isinstance(body, dict):
raise ValueError("Request body is required")
username = body.get("username")
old_password = body.get("oldPassword")
new_password = body.get("newPassword")
if not username or not old_password or not new_password:
raise ValueError("username, oldPassword, and newPassword are required")
if len(new_password) < 8:
raise ValueError("New password must be at least 8 characters")
update_password(username, old_password, new_password)
logger.info("Password changed for user %r", username)
return {"ok": True}
@registry.register(GET_AUTH_USERS)
def auth_list_users(_request: Any, body: Any) -> list[dict[str, Any]]:
"""List all users.
Args:
_request: Unused.
body: Unused.
Returns:
List of user summary dicts.
"""
return list_users()
@registry.register(POST_AUTH_USER_CREATE)
def auth_create_user(_request: Any, body: Any) -> dict[str, Any]:
"""Create a new user.
Args:
_request: Unused.
body: Dict with ``username``, ``password``, ``permissions``.
Returns:
Created user dict.
Raises:
ConflictError: If user already exists.
"""
if not body or not isinstance(body, dict):
raise ValueError("Request body is required")
username = body.get("username")
password = body.get("password")
permissions = body.get("permissions", {})
if not username or not password:
raise ValueError("username and password are required")
if len(password) < 8:
raise ValueError("Password must be at least 8 characters")
try:
user = create_user(username, password, permissions)
return {
"id": user["id"],
"username": user["username"],
"permissions": user["permissions"],
}
except ValueError as e:
raise ConflictError(str(e)) from e
@registry.register(POST_AUTH_USER_UPDATE)
def auth_update_user(_request: Any, body: Any) -> dict[str, Any]:
"""Update a user's permissions.
Args:
_request: Unused.
body: Dict with optional ``permissions``, ``password`` keys.
Path param ``username`` is merged into body by the daemon.
Returns:
Updated user dict.
Raises:
NotFoundError: If user not found.
"""
if not body or not isinstance(body, dict):
raise ValueError("Request body is required")
username = body.get("username")
if not username:
raise ValueError("username is required")
user = get_user(username)
if user is None:
raise NotFoundError(f"User {username!r} not found")
if "permissions" in body:
update_permissions(username, body["permissions"])
user = get_user(username)
if user is None:
raise NotFoundError(f"User {username!r} not found")
return {
"id": user["id"],
"username": user["username"],
"permissions": user["permissions"],
}
@registry.register(DELETE_AUTH_USER)
def auth_delete_user(_request: Any, body: Any) -> dict[str, Any]:
"""Delete a user.
Self-deletion is blocked by Flask middleware (auth blueprint).
Args:
_request: Unused.
body: Path param ``username`` merged by the daemon.
Returns:
Success response.
Raises:
NotFoundError: If user not found.
"""
if not body or not isinstance(body, dict):
raise ValueError("Request body is required")
username = body.get("username")
if not username:
raise ValueError("username is required")
try:
delete_user(username)
logger.info("User %r deleted", username)
return {"ok": True}
except ValueError as e:
if "not found" in str(e):
raise NotFoundError(str(e)) from e
raise
# ---------------------------------------------------------------------------
# WebAuthn handlers
# ---------------------------------------------------------------------------
def _check_webauthn_domain(body: Any) -> tuple[str, str]:
"""Validate that the request domain is eligible for WebAuthn.
Returns (origin, rp_id) if valid. Raises ValueError otherwise.
"""
if not body or not isinstance(body, dict):
raise ValueError("Request body is required")
if not webauthn_is_enabled():
raise ValueError("WebAuthn is disabled")
rp_id = body.get("webauthn_rp_id")
origin = body.get("webauthn_origin")
if not rp_id or not origin:
raise ValueError("Missing WebAuthn domain configuration")
if rp_id not in get_management_domains():
raise ValueError("WebAuthn is not available on this domain")
return origin, rp_id
@registry.register(GET_AUTH_WEBAUTHN_CAPABLE)
def webauthn_capable(_request: Any, body: Any) -> dict[str, Any]:
"""Check if WebAuthn is available for the current request domain."""
rp_id = body.get("webauthn_rp_id") if body else None
origin = body.get("webauthn_origin") if body else None
if not webauthn_is_enabled():
return {"enabled": False, "reason": "WebAuthn is disabled in config"}
if not rp_id or not origin:
return {"enabled": False, "reason": "Domain information unavailable"}
mgmt_domains = get_management_domains()
if rp_id not in mgmt_domains:
return {"enabled": False, "reason": "Not a management domain"}
return {
"enabled": True,
"rp_id": rp_id,
"rp_name": get_rp_name(),
"origin": origin,
}
@registry.register(POST_AUTH_WEBAUTHN_REGISTER_BEGIN)
def webauthn_register_begin(_request: Any, body: Any) -> dict[str, Any]:
"""Begin WebAuthn registration — return options for ``credentials.create()``."""
origin, rp_id = _check_webauthn_domain(body)
username = body.get("username")
if not username:
raise ValueError("username is required")
options = create_registration_options(
username,
origin=origin,
rp_id=rp_id,
)
return options
@registry.register(POST_AUTH_WEBAUTHN_REGISTER_FINISH)
def webauthn_register_finish(_request: Any, body: Any) -> dict[str, Any]:
"""Finish WebAuthn registration — verify credential and persist."""
origin, rp_id = _check_webauthn_domain(body)
username = body.get("username")
credential_response = body.get("credential_response")
registration_options = body.get("registration_options")
credential_name = body.get("name", "")
if not username or not credential_response or not registration_options:
raise ValueError(
"username, credential_response, and registration_options are required"
)
cred = verify_registration(
username,
credential_response,
registration_options,
credential_name,
origin=origin,
rp_id=rp_id,
)
return {"ok": True, "credential": cred}
@registry.register(POST_AUTH_WEBAUTHN_AUTHENTICATE_BEGIN)
def webauthn_authenticate_begin(_request: Any, body: Any) -> dict[str, Any]:
"""Begin WebAuthn authentication — return options for ``credentials.get()``.
Public endpoint — no JWT required. Returns ``{"noWebAuthn": true}`` if the
user has no registered credentials (so the frontend can fall back to password).
"""
_, rp_id = _check_webauthn_domain(body)
username = body.get("username")
if not username:
raise ValueError("username is required")
options = create_authentication_options(
username,
rp_id=rp_id,
)
if options is None:
return {"no_webauthn": True}
return options
@registry.register(POST_AUTH_WEBAUTHN_AUTHENTICATE_FINISH)
def webauthn_authenticate_finish(_request: Any, body: Any) -> dict[str, Any]:
"""Finish WebAuthn authentication — verify assertion, issue tokens.
Public endpoint — no JWT required.
"""
origin, rp_id = _check_webauthn_domain(body)
username = body.get("username")
assertion_response = body.get("assertion_response")
auth_options = body.get("auth_options")
client_ip = body.get("client_ip")
if not username or not assertion_response or not auth_options:
raise ValueError("username, assertion_response, and auth_options are required")
if not check_webauthn_rate(username, client_ip):
raise ValueError("Too many WebAuthn attempts. Please try again later.")
try:
verify_authentication(
username,
assertion_response,
auth_options,
origin=origin,
rp_id=rp_id,
)
except ValueError:
record_webauthn_failure(username, client_ip)
raise
record_webauthn_success(username, client_ip)
user = get_user(username)
if user is None:
raise ValueError("User not found")
permissions = user["permissions"]
tokens = generate_tokens(username, permissions)
return {
"tokens": tokens,
"access_ttl": get_access_ttl(),
"user": {
"id": user["id"],
"username": user["username"],
},
"permissions": permissions,
}
@registry.register(GET_AUTH_WEBAUTHN_CREDENTIALS)
def webauthn_credentials(request: Any, body: Any) -> list[dict[str, Any]]:
"""List WebAuthn credentials for the authenticated user."""
username = body.get("username") if body else None
if not username:
raise ValueError("No active session")
return list_credentials(username)
@registry.register(DELETE_AUTH_WEBAUTHN_CREDENTIAL)
def webauthn_remove_credential(request: Any, body: Any) -> dict[str, Any]:
"""Remove a WebAuthn credential."""
if not body or not isinstance(body, dict):
raise ValueError("Request body is required")
username = body.get("username")
if not username:
raise ValueError("No active session")
credential_id = body.get("credential_id")
if not credential_id:
raise ValueError("credential_id is required")
try:
remove_credential(username, credential_id)
logger.info("WebAuthn credential removed for %s", username)
return {"ok": True}
except ValueError as e:
if "not found" in str(e):
raise NotFoundError(str(e)) from e
raise
@registry.register(GET_AUTH_WEBAUTHN_CREDENTIAL_COUNTS)
def webauthn_credential_counts(_request: Any, body: Any) -> dict[str, int]:
"""Return credential counts for all users (admin endpoint)."""
return get_all_credential_counts()
+26
View File
@@ -0,0 +1,26 @@
"""Shared helpers for daemon handlers."""
from typing import Any
from daemon.server import refresh_state
from lib.sync import SyncEvent, bus
def emit_and_refresh(
subsystem: str, payload: dict[str, Any] | None = None
) -> list[str]:
"""Emit a ``config_saved`` sync event and refresh the affected state.
All mutation handlers end with the same tail: emit the event, refresh
the source subsystem plus every subsystem the sync touched.
Args:
subsystem: Source subsystem name.
payload: Event payload (e.g. ``{"action": "zone_created"}``).
Returns:
Subsystems affected by the sync event.
"""
sync_result = bus.emit(SyncEvent(subsystem, "config_saved", payload or {}))
refresh_state([subsystem, *sync_result.affected_subsystems])
return sync_result.affected_subsystems
+61 -31
View File
@@ -8,6 +8,7 @@ from typing import Any
from jinja2 import Environment, FileSystemLoader
from daemon.handlers.common import emit_and_refresh
from daemon.iface import (
DELETE_DNSMASQ_DNS_RECORD_REMOVE,
DELETE_DNSMASQ_RANGES_REMOVE,
@@ -24,8 +25,17 @@ from daemon.iface import (
POST_DNSMASQ_STATIC_LEASE_ADD,
POST_DNSMASQ_UPSTREAMS,
)
from daemon.server import NotFoundError, refresh_state, registry
from lib.common import deep_merge, ensure_dirs, load_json, run, run_proc, save_json
from daemon.server import NotFoundError, registry
from lib.common import (
deep_merge,
ensure_dirs,
get_interface_ip,
load_json,
run,
save_json,
stamp_applied,
strip_apply_meta,
)
logger = logging.getLogger(__name__)
@@ -35,7 +45,7 @@ DATA_DIR = PROJECT_DIR / "data" / "dnsmasq"
CONFIG_PATH = CONFIG_DIR / "config.json"
FRAGMENTS_DIR = DATA_DIR / "fragments"
DNSMASQ_CONF = "/etc/dnsmasq.d/vacuum-wall.conf"
LEASE_FILE = "/var/lib/dnsmasq/dnsmasq.leases"
LEASE_FILE = "/var/lib/misc/dnsmasq.leases"
ENV = Environment(
loader=FileSystemLoader(str(PROJECT_DIR / "system")),
@@ -128,7 +138,8 @@ def get_config(_request: Any, _body: Any) -> dict[str, Any]:
dm = _get_dnsmasq_state()
if dm:
return dm.get("config", {})
return _get_config()
cfg = _get_config()
return strip_apply_meta(cfg)
@registry.register(POST_DNSMASQ_CONFIG)
@@ -141,7 +152,7 @@ def save_config_handler(_request: Any, body: dict[str, Any] | None) -> dict[str,
if not body:
raise ValueError("Request body required")
_save_config(body)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "config_saved"})
return {"config_saved": True}
@@ -157,7 +168,7 @@ def patch_config(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
current = _get_config()
merged = deep_merge(current, body)
_save_config(merged)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "config_patched"})
return {"config_saved": True}
@@ -172,16 +183,20 @@ def apply_config(_request: Any, _body: Any) -> dict[str, Any]:
conf_text = _generate_conf(cfg)
ensure_dirs(CONFIG_DIR, DATA_DIR, FRAGMENTS_DIR)
run(["mkdir", "-p", "/etc/dnsmasq.d"], sudo=True)
run_proc(
["tee", DNSMASQ_CONF, "--"],
sudo=True,
check=True,
input=conf_text,
)
run(["systemctl", "reload", "dnsmasq"], sudo=True)
logger.info("dnsmasq config written and reloaded")
refresh_state(["dnsmasq"])
return {"applied": True}
tmp = Path("/run/vacuum-wall/dnsmasq.tmp")
tmp.parent.mkdir(exist_ok=True)
tmp.write_text(conf_text)
run(["cp", "--", str(tmp), str(DNSMASQ_CONF)], sudo=True)
tmp.unlink(missing_ok=True)
run(["systemctl", "restart", "dnsmasq"], sudo=True)
logger.info("dnsmasq config written and restarted")
# Store the applied config snapshot + hash so the state collector can
# detect pending changes and report what specifically changed.
cfg_after = _get_config()
stamp_applied(cfg_after)
_save_config(cfg_after)
synced = emit_and_refresh("dnsmasq", {"action": "config_applied"})
return {"applied": True, "synced": synced}
@registry.register(GET_DNSMASQ_STATUS)
@@ -203,15 +218,30 @@ def set_dhcp_range(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]
Endpoint: POST /dnsmasq/ranges/add
Add or update a DHCP pool range by interface. Raises ValueError on invalid input.
Auto-populates ``gateway`` from the interface's IPv4 address when not provided.
"""
if not body:
raise ValueError("Request body required")
iface = body.get("interface", "").strip() or ""
iface = (body.get("interface") or "").strip()
start = body.get("start", "").strip()
end = body.get("end", "").strip()
lease_time = body.get("lease_time", "12h")
if not start or not end:
raise ValueError("'start' and 'end' are required")
# Resolve gateway: explicit value > existing range value > interface IP
gateway = body.get("gateway")
if not gateway:
# Check existing range for same interface
cfg_tmp = _get_config()
for r in cfg_tmp["dhcp"]["ranges"]:
if r.get("interface") == iface:
gateway = r.get("gateway")
break
# Fall back to interface's own IP
if not gateway and iface:
gateway = get_interface_ip(iface)
cfg = _get_config()
ranges = cfg["dhcp"]["ranges"]
found = False
@@ -223,8 +253,8 @@ def set_dhcp_range(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]
"end": end,
"lease_time": lease_time,
}
if body.get("gateway"):
ranges[i]["gateway"] = body["gateway"]
if gateway:
ranges[i]["gateway"] = gateway
if body.get("dns"):
ranges[i]["dns"] = body["dns"]
found = True
@@ -236,13 +266,13 @@ def set_dhcp_range(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]
"end": end,
"lease_time": lease_time,
}
if body.get("gateway"):
entry["gateway"] = body["gateway"]
if gateway:
entry["gateway"] = gateway
if body.get("dns"):
entry["dns"] = body["dns"]
ranges.append(entry)
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "range_added", "interface": iface})
return {"interface": iface, "start": start, "end": end}
@@ -277,7 +307,7 @@ def remove_dhcp_range(_request: Any, body: dict[str, Any] | None) -> dict[str, A
f"DHCP range for interface '{iface}' ({start}-{end}) not found"
)
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "range_removed", "interface": iface})
return {"interface": iface, "start": start, "end": end}
@@ -316,14 +346,14 @@ def add_static_lease(_request: Any, body: dict[str, Any] | None) -> dict[str, An
if hostname is not None:
leases[i]["hostname"] = hostname
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "static_lease_added", "mac": mac})
return {"mac": mac, "ip": ip, "hostname": hostname}
entry: dict[str, Any] = {"mac": mac, "ip": ip}
if hostname:
entry["hostname"] = hostname
leases.append(entry)
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "static_lease_added", "mac": mac})
return {"mac": mac, "ip": ip, "hostname": hostname}
@@ -348,7 +378,7 @@ def remove_static_lease(_request: Any, body: dict[str, Any] | None) -> dict[str,
if len(cfg["dhcp"]["static_leases"]) == before:
raise NotFoundError(f"Static lease for MAC '{mac}' not found")
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "static_lease_removed", "mac": mac})
return {"mac": mac}
@@ -374,14 +404,14 @@ def add_dns_record(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]
if hostname is not None:
records[i]["hostname"] = hostname
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "dns_record_added", "name": name})
return {"name": name, "address": address, "hostname": hostname}
entry: dict[str, Any] = {"name": name, "address": address}
if hostname:
entry["hostname"] = hostname
records.append(entry)
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "dns_record_added", "name": name})
return {"name": name, "address": address, "hostname": hostname}
@@ -404,7 +434,7 @@ def remove_dns_record(_request: Any, body: dict[str, Any] | None) -> dict[str, A
if len(cfg["dns"]["custom_records"]) == before:
raise NotFoundError(f"DNS record '{name}' not found")
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "dns_record_removed", "name": name})
return {"name": name}
@@ -420,7 +450,7 @@ def set_upstreams(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
cfg = _get_config()
cfg["dns"]["upstreams"] = list(body["servers"])
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "upstreams_set"})
return {"upstreams": cfg["dns"]["upstreams"]}
@@ -437,5 +467,5 @@ def set_domain(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
cfg = _get_config()
cfg["dns"]["domain"] = domain if domain else None
_save_config(cfg)
refresh_state(["dnsmasq"])
emit_and_refresh("dnsmasq", {"action": "domain_set"})
return {"domain": cfg["dns"]["domain"]}
+581 -58
View File
@@ -10,6 +10,7 @@ from pathlib import Path
from typing import Any
from uuid import uuid4
from daemon.handlers.common import emit_and_refresh
from daemon.iface import (
DELETE_FIREWALL_FORWARD_PORT_REMOVE,
DELETE_FIREWALL_RICH_RULES_REMOVE,
@@ -33,12 +34,17 @@ from daemon.iface import (
POST_FIREWALL_ZONES_INTERFACES,
POST_FIREWALL_ZONES_SERVICES,
)
from daemon.server import NotFoundError, refresh_state, registry
from lib.common import load_json, run, save_json
from daemon.server import ConflictError, NotFoundError, registry
from lib import network
from lib.common import load_json, run, save_json, stamp_applied, strip_apply_meta
from lib.firewall import (
_normalize_target,
_now_iso,
_parse_active_zones,
_parse_all_zones_output,
_parse_zone_output,
fw_change_summary,
validate_coverage,
)
from lib.firewall import (
save_backup as _save_backup,
@@ -78,11 +84,65 @@ def _save_config(cfg: dict[str, Any]) -> None:
save_json(CONFIG_FILE, cfg, indent=2)
def _check_coverage(cfg: dict[str, Any]) -> None:
"""Reject a config that leaves a managed interface without coverage.
Runs the pure ``validate_coverage`` invariant against the current
network config. ``lo`` and ``wg*`` are exempt, and interfaces declared
in the top-level ``unmanaged`` list are exempt.
Args:
cfg: The (merged or full) firewall config dict to validate.
Raises:
ValueError: If a network-managed interface is not covered by any
zone and is not declared under ``unmanaged``.
"""
uncovered = validate_coverage(cfg, network.get_config())
if uncovered:
raise ValueError(
"Refusing to save: "
f"{', '.join(repr(n) for n in uncovered)} "
f"have no firewall zone coverage and are not declared in the "
f"'unmanaged' list. Assign each interface to a zone, or add it "
f"to the top-level 'unmanaged' list."
)
def _reload() -> None:
"""Reload firewalld to apply permanent changes."""
run(["firewall-cmd", "--reload"], sudo=True)
def _default_zone() -> str:
"""Return the firewalld default zone name.
The default zone is the catch-all for any interface without an explicit
zone assignment (including VPN interfaces), so it normally fronts the WAN.
"""
return run(["firewall-cmd", "--get-default-zone"], sudo=True).strip()
def _would_remove_mgmt(zone: str, services: list[str]) -> bool:
"""Return True if *services* lacks both https and ssh on the default zone.
The default zone fronts unassigned (WAN/VPN) interfaces, so removing both
management access (https via nginx) and remote recovery (ssh) from it
would leave no path back except a physical console.
"""
if "https" in services or "ssh" in services:
return False
try:
default = _default_zone()
except Exception:
logger.warning(
"Could not determine the firewalld default zone; failing closed for %s",
zone,
)
return True
return zone == default
def _fp_to_str(fp: dict[str, Any]) -> str:
"""Convert a forward-port dict to firewall-cmd CLI argument string."""
parts = [f"port={fp['port']}", f"proto={fp['proto']}"]
@@ -104,39 +164,104 @@ def _get_forward_ports(zone_name: str) -> list[str]:
return []
def _config_apply() -> dict[str, Any]:
"""Apply the declarative config to live firewalld."""
def _config_apply(force: bool = False) -> dict[str, Any]:
"""Apply saved declarative config to live firewalld.
For each zone in the config, reconciles interfaces, services, target,
masquerade, rich rules, and forward ports by removing old values first,
then adding desired values. Reloads firewalld at the end.
With *force* False (default), a ``ConflictError`` is raised before any
mutation in two cases; pass ``force=True`` to override either:
- the config would strip both https and ssh from the default zone
(management lockout);
- the config leaves a network-subsystem-managed interface with no
firewall zone coverage (``lo`` and ``wg*`` interfaces are excluded).
The config is the source of truth for zone interfaces — an absent
``interfaces`` key counts as empty — so coverage is computed from the
config alone via ``validate_coverage`` with no live-state fallback.
Interfaces listed in the top-level ``unmanaged`` key are exempt. The
same invariant is enforced at save time (POST/PATCH /firewall/config),
so a conflict here means the network config changed after the firewall
config was saved (e.g. a new interface no zone covers).
"""
from lib.firewall import get_config as _get_lib_config
cfg = _get_lib_config()
cfg_zones = cfg.get("zones", {})
full_state: dict[str, Any] = {
"active_zones": {},
"interfaces": [],
"available_services": [],
"zones": {},
"rich_rules": {},
"timestamp": "",
}
_save_backup(full_state)
if not force:
default_zone = _default_zone()
lockout_zones = [
zn
for zn, zc in cfg_zones.items()
if zn == default_zone
and "https" not in zc.get("services", [])
and "ssh" not in zc.get("services", [])
]
if lockout_zones:
raise ConflictError(
f"Refusing to remove both https and ssh from default zone(s) "
f"{', '.join(repr(z) for z in lockout_zones)}: management access "
f"and remote recovery would be lost. Add at least one of them "
f'to the zone\'s services, or pass {{"force": true}}.'
)
# Coverage invariant: every network-managed interface must be
# covered by a zone in the config (or declared unmanaged), or
# traffic (and DHCP) on that segment is dropped. Pure config check
# — the config is the source of truth, so no live-state comparison.
uncovered = validate_coverage(cfg, network.get_config())
if uncovered:
raise ConflictError(
"Refusing to apply: "
f"{', '.join(repr(n) for n in uncovered)} "
f"have no firewall zone coverage in the config and are not "
f"declared unmanaged, so all traffic (including DHCP) from "
f"those segments would be dropped. Assign each interface to "
f"a zone (or list it under the config's top-level 'unmanaged' "
f'key), or pass {{"force": true}}.'
)
# Pre-apply snapshot for disaster recovery: the permanent zone view plus
# the declarative config, captured before any mutation. The permanent
# view is what is reproducible for manual recovery.
backup_path = _save_backup(
{
"timestamp": _now_iso(),
"default_zone": _default_zone(),
"zones": _parse_all_zones_output(
run(["firewall-cmd", "--list-all-zones", "--permanent"], sudo=True)
),
"config": cfg,
}
)
available = run(["firewall-cmd", "--get-zones"], sudo=True).split()
applied: list[str] = []
for zone_name, zone_cfg in cfg_zones.items():
# Two-step reconciliation: remove current values, then add desired values.
# This ensures idempotency — running apply twice produces the same result.
need_create = zone_name not in available
if need_create:
target = _normalize_target(zone_cfg.get("target", "DEFAULT"))
# Create new zone first (--new-zone is required before --set-target)
run(
[
"firewall-cmd",
f"--zone={zone_name}",
f"--set-target={target}",
"--permanent",
],
["firewall-cmd", f"--new-zone={zone_name}", "--permanent"],
sudo=True,
)
target = _normalize_target(zone_cfg.get("target", "DEFAULT"))
if target != "default":
run(
[
"firewall-cmd",
f"--zone={zone_name}",
f"--set-target={target}",
"--permanent",
],
sudo=True,
)
_reload()
else:
desired_target = _normalize_target(zone_cfg.get("target", "DEFAULT"))
@@ -153,6 +278,9 @@ def _config_apply() -> dict[str, Any]:
check=False,
)
# Step 2: Reconcile services — remove all current, add desired list.
# Firewall-cmd doesn't have a "set-services" bulk operation, so we
# remove each existing service and then add each desired service.
current_svcs: list[str] = []
with suppress(Exception):
current_svcs = _parse_zone_output(
@@ -181,6 +309,10 @@ def _config_apply() -> dict[str, Any]:
sudo=True,
)
# Step 3: Reconcile interfaces — same remove-then-add pattern.
# The config is the source of truth: an absent "interfaces" key
# counts as an empty list (unassign-all), matching the coverage
# invariant and the pending diff.
current_ifaces: list[str] = []
with suppress(Exception):
current_ifaces = _parse_zone_output(
@@ -209,12 +341,34 @@ def _config_apply() -> dict[str, Any]:
sudo=True,
)
mq = zone_cfg.get("masquerade", False)
if mq is not None:
action = "--add-masquerade" if mq else "--remove-masquerade"
# Step 4: Toggle masquerade if explicitly set (None means "don't change").
# Skip 'public' — Step 7 handles masquerade propagation for nftables.
if zone_name != "public":
mq = zone_cfg.get("masquerade", False)
if mq is not None:
action = "--add-masquerade" if mq else "--remove-masquerade"
run(
["firewall-cmd", f"--zone={zone_name}", action, "--permanent"],
sudo=True,
)
# Step 5: Reconcile rich rules — remove all current, add desired.
# Note: firewall-cmd doesn't track rule IDs for rich rules, so we
# must remove by full rule string match.
current_rules = _parse_zone_output(
zone_name,
run(["firewall-cmd", f"--zone={zone_name}", "--list-all"], sudo=True),
).get("rich-rules", [])
for rule_str in current_rules:
run(
["firewall-cmd", f"--zone={zone_name}", action, "--permanent"],
[
"firewall-cmd",
f"--zone={zone_name}",
f"--remove-rich-rule={rule_str}",
"--permanent",
],
sudo=True,
check=False,
)
for rule_entry in zone_cfg.get("rich_rules", []):
@@ -235,6 +389,7 @@ def _config_apply() -> dict[str, Any]:
check=False,
)
# Step 6: Reconcile forward ports — same pattern.
current_fps = _get_forward_ports(zone_name)
for fp_str in current_fps:
run(
@@ -262,16 +417,39 @@ def _config_apply() -> dict[str, Any]:
applied.append(zone_name)
# Step 7: Ensure masquerade propagation for nftables backend.
# With firewalld's nftables backend, POSTROUTING policy chains route traffic
# to the OUTPUT interface's zone chain. Traffic from internal zones (eth1)
# exiting through public (eth0) hits public's POSTROUTING chain, not
# internal's. If any non-public zone has masquerade enabled but the public
# zone doesn't, NAT silently fails — so propagate masquerade to public.
_any_non_public_mq = any(
z.get("masquerade", False) for zn, z in cfg_zones.items() if zn != "public"
)
_public_mq = cfg_zones.get("public", {}).get("masquerade", False)
if _any_non_public_mq and not _public_mq:
logger.info("Propagating masquerade to public zone for nftables compatibility")
run(
["firewall-cmd", "--zone=public", "--add-masquerade", "--permanent"],
sudo=True,
)
cfg.setdefault("zones", {}).setdefault("public", {})["masquerade"] = True
_save_config(cfg)
elif not _any_non_public_mq and _public_mq:
logger.info("No non-public zone needs masquerade, removing from public zone")
run(
["firewall-cmd", "--zone=public", "--remove-masquerade", "--permanent"],
sudo=True,
)
cfg.setdefault("zones", {}).setdefault("public", {})["masquerade"] = False
_save_config(cfg)
_reload()
full_state = {
"active_zones": {},
"interfaces": [],
"available_services": [],
"zones": {},
"rich_rules": {},
"timestamp": "",
}
backup_path = _save_backup(full_state)
# Record the applied config snapshot + hash so pending-changes detection
# and cancel/revert work like the hash-based subsystems.
applied_cfg = _get_config()
stamp_applied(applied_cfg)
_save_config(applied_cfg)
logger.info("Firewall config applied to %d zones", len(applied))
return {
"applied_zones": applied,
@@ -301,6 +479,16 @@ def get_interfaces(_request: Any, _body: Any) -> list[dict[str, Any]]:
@registry.register(GET_FIREWALL_ZONES)
def get_zones(_request: Any, _body: Any) -> dict[str, Any]:
"""Return active + available zones from state store.
Args:
_request: The incoming HTTP request (unused).
_body: The request body (unused).
Returns:
Dict with ``active`` (zone→interfaces mapping) and
``available`` (list of all zone names).
"""
fw = _get_fw_state()
active = fw.get("active_zones", {})
zones = fw.get("zones", {})
@@ -309,6 +497,19 @@ def get_zones(_request: Any, _body: Any) -> dict[str, Any]:
@registry.register(GET_FIREWALL_ZONES_INFO)
def get_zone_info(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Return zone config by name; ``NotFoundError`` if absent.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone`` key (zone name).
Returns:
Zone config dict.
Raises:
ValueError: If ``zone`` key is missing from body.
NotFoundError: If the specified zone does not exist.
"""
if not body or "zone" not in body:
raise ValueError("'zone' is required")
zone = body["zone"]
@@ -321,6 +522,15 @@ def get_zone_info(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
@registry.register(GET_FIREWALL_ZONES_ALL)
def get_all_zones_info(_request: Any, _body: Any) -> list[dict[str, Any]]:
"""Return list of all active zone configs.
Args:
_request: The incoming HTTP request (unused).
_body: The request body (unused).
Returns:
List of zone config dicts for all active zones.
"""
fw = _get_fw_state()
active = fw.get("active_zones", {})
zones = fw.get("zones", {})
@@ -333,57 +543,170 @@ def get_all_zones_info(_request: Any, _body: Any) -> list[dict[str, Any]]:
@registry.register(GET_FIREWALL_SERVICES)
def get_services(_request: Any, _body: Any) -> list[str]:
"""Return list of available firewall services.
Args:
_request: The incoming HTTP request (unused).
_body: The request body (unused).
Returns:
List of service names available in firewalld.
"""
fw = _get_fw_state()
return fw.get("available_services", [])
@registry.register(GET_FIREWALL_CONFIG)
def get_config(_request: Any, _body: Any) -> dict[str, Any]:
return _get_config()
"""Return declarative config from JSON store.
Args:
_request: The incoming HTTP request (unused).
_body: The request body (unused).
Returns:
Full firewall config dict (apply bookkeeping keys stripped).
"""
return strip_apply_meta(_get_config())
@registry.register(POST_FIREWALL_CONFIG)
def save_config_handler(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Save the full declarative firewall configuration.
Validates that the request body contains a ``zones`` dict, persists it
to the JSON store, emits a ``config_saved`` sync event to the
cross-subsystem sync bus, and refreshes the state store.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with a ``zones`` dict mapping zone names to
zone configurations.
Returns:
Dict with ``config_saved`` flag set to ``True``.
Raises:
ValueError: If body is empty, missing ``zones`` key, ``zones`` is
not a dict, ``unmanaged`` is not a list, or the config leaves a
network-managed interface without zone coverage.
"""
if not body or "zones" not in body:
raise ValueError("'zones' key is required")
if not isinstance(body["zones"], dict):
raise ValueError("'zones' must be a dict")
if "unmanaged" in body and not isinstance(body["unmanaged"], list):
raise ValueError("'unmanaged' must be a list")
_check_coverage(body)
_save_config(body)
logger.info("Firewall config saved (%d zones)", len(body["zones"]))
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "config_saved"})
return {"config_saved": True}
@registry.register(PATCH_FIREWALL_CONFIG)
def patch_config(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Deep-merge body into current config, save, emit sync event, refresh state.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with partial config to merge.
Returns:
Dict with ``config_saved`` flag set to ``True``.
Raises:
ValueError: If body is empty, ``unmanaged`` is not a list, or the
merged config leaves a network-managed interface without zone
coverage.
"""
if not body:
raise ValueError("Request body must be a JSON object")
if "unmanaged" in body and not isinstance(body["unmanaged"], list):
raise ValueError("'unmanaged' must be a list")
from lib.common import deep_merge
current = _get_config()
merged = deep_merge(current, body)
_check_coverage(merged)
_save_config(merged)
logger.info("Firewall config patched: %s", sorted(body.keys()))
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "config_patched"})
return {"config_saved": True}
@registry.register(GET_FIREWALL_CONFIG_PENDING)
def config_pending_handler(_request: Any, _body: Any) -> dict[str, Any]:
"""Return pending changes with human-readable summaries.
Compares saved config against live firewalld state and returns a list
of differences.
Args:
_request: The incoming HTTP request (unused).
_body: The request body (unused).
Returns:
Dict with ``pending`` (list of change dicts) and ``pending_summary``
(human-readable strings).
"""
fw = _get_fw_state()
return fw.get("pending", {})
pending = fw.get("pending", {})
changes = pending.get("pending", [])
summaries = [
fw_change_summary(c.get("zone", "unknown"), c.get("type", "unknown"), c)
for c in changes
]
return {**pending, "pending_summary": summaries}
@registry.register(POST_FIREWALL_CONFIG_APPLY)
def config_apply(_request: Any, _body: Any) -> dict[str, Any]:
result = _config_apply()
"""Apply pending config to live firewalld, emit sync event, refresh state.
Args:
_request: The incoming HTTP request (unused).
_body: Optional JSON body; ``{"force": true}`` overrides the
management-lockout guard and the interface-coverage guard.
Returns:
Dict with ``applied_zones`` (list of zone names), ``backup`` (path),
and ``synced`` (affected subsystems).
Raises:
ConflictError: If the config would strip both https and ssh from the
default zone, or would remove zone coverage from a
network-managed interface that is covered now, and ``force`` is
not set.
"""
force = bool(_body and _body.get("force"))
result = _config_apply(force=force)
logger.info("Firewall config applied: %s", result.get("applied_zones", []))
refresh_state(["firewall"])
synced = emit_and_refresh("firewall", {"action": "config_applied"})
result["synced"] = synced
return result
@registry.register(POST_FIREWALL_ZONES_CREATE)
def create_zone(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Create a new firewall zone, emit sync event, refresh state.
Runs ``--new-zone`` first (required before ``--set-target``), then sets
the target only when it normalizes to something other than ``default``
(the implicit firewalld target is never re-set), then reloads.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``name`` (zone name) and optional ``target``.
Returns:
Dict with ``zone`` key set to the zone name.
Raises:
ValueError: If body is empty, missing name, or zone already exists.
"""
if not body:
raise ValueError("Request body required")
zone_name = body.get("name", "").strip()
@@ -393,23 +716,42 @@ def create_zone(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
available = run(["firewall-cmd", "--get-zones"], sudo=True).split()
if zone_name in available:
raise ValueError(f"Zone '{zone_name}' already exists")
run(
[
"firewall-cmd",
f"--zone={zone_name}",
f"--set-target={target}",
"--permanent",
],
sudo=True,
)
# Create the zone first; --set-target requires the zone to exist.
run(["firewall-cmd", f"--new-zone={zone_name}", "--permanent"], sudo=True)
# "default" is firewalld's implicit target and cannot be meaningfully
# re-set, so only explicit ACCEPT/DROP/REJECT targets are applied.
normalized_target = _normalize_target(target)
if normalized_target != "default":
run(
[
"firewall-cmd",
f"--zone={zone_name}",
f"--set-target={normalized_target}",
"--permanent",
],
sudo=True,
)
_reload()
logger.info("Zone '%s' created (target=%s)", zone_name, target)
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "zone_created", "zone": zone_name})
return {"zone": zone_name}
@registry.register(DELETE_FIREWALL_ZONES_DELETE)
def delete_zone(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Delete zone via firewall-cmd, emit sync event, refresh state.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone`` (zone name).
Returns:
Dict with ``zone`` key set to the zone name.
Raises:
ValueError: If body is empty or missing ``zone``.
NotFoundError: If the zone does not exist.
"""
if not body or "zone" not in body:
raise ValueError("'zone' is required")
zone = body["zone"]
@@ -419,12 +761,30 @@ def delete_zone(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
run(["firewall-cmd", f"--zone={zone}", "--delete", "--permanent"], sudo=True)
_reload()
logger.info("Zone '%s' deleted", zone)
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "zone_deleted", "zone": zone})
return {"zone": zone}
@registry.register(POST_FIREWALL_ZONES_INTERFACES)
def set_zone_interfaces(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Replace zone interfaces, reassigning interfaces from old zones.
When the new selection leaves an interface in no zone at all, a
prominent warning is logged (clients on that segment lose connectivity
and DHCP); the operation is not blocked since it is a deliberate UI
action.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone`` and ``interfaces`` list.
Returns:
Dict with ``zone`` and ``interfaces`` keys.
Raises:
ValueError: If body is missing ``zone``.
NotFoundError: If the zone does not exist.
"""
if not body:
raise ValueError("Request body required")
zone = body.get("zone", "").strip()
@@ -468,6 +828,20 @@ def set_zone_interfaces(_request: Any, body: dict[str, Any] | None) -> dict[str,
sudo=True,
)
# Flag interfaces that ended up in no zone at all — clients on those
# segments lose connectivity (including DHCP).
for iface in set(active.get(zone, [])) - set(interfaces):
if not any(
iface in az_ifaces for az, az_ifaces in active.items() if az != zone
):
logger.warning(
"Interface '%s' is now in NO firewall zone: clients on that "
"segment will lose connectivity and DHCP (zone '%s' no longer "
"covers it).",
iface,
zone,
)
_reload()
# Update config
@@ -486,15 +860,34 @@ def set_zone_interfaces(_request: Any, body: dict[str, Any] | None) -> dict[str,
old_zone_cfg["interfaces"] = new_ifaces
elif "interfaces" in old_zone_cfg:
del old_zone_cfg["interfaces"]
# This mutation already applied to live firewalld, so re-stamp the applied
# baseline: cancel-all must revert to this state, not an older snapshot.
stamp_applied(cfg)
_save_config(cfg)
logger.info("Zone '%s' interfaces set to %s", zone, interfaces)
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "interfaces_set", "zone": zone})
return {"zone": zone, "interfaces": interfaces}
@registry.register(POST_FIREWALL_ZONES_SERVICES)
def set_zone_services(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Replace zone services, persist to declarative config, and refresh state.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone`` and ``services`` list; optional
``force`` (bool) overrides the management-lockout guard.
Returns:
Dict with ``zone`` and ``services`` keys.
Raises:
ValueError: If body is missing ``zone``.
NotFoundError: If the specified zone does not exist.
ConflictError: If the change would strip both https and ssh from the
firewalld default zone and ``force`` is not set.
"""
if not body:
raise ValueError("Request body required")
zone = body.get("zone", "").strip()
@@ -503,6 +896,12 @@ def set_zone_services(_request: Any, body: dict[str, Any] | None) -> dict[str, A
raise ValueError("'zone' is required")
if zone not in run(["firewall-cmd", "--get-zones"], sudo=True).split():
raise NotFoundError(f"Zone '{zone}' does not exist")
if not body.get("force") and _would_remove_mgmt(zone, list(services)):
raise ConflictError(
f"Refusing to remove both https and ssh from default zone '{zone}': "
f"management access and remote recovery would be lost. Add at least "
f'one of them back, or send "force": true to override.'
)
current = _parse_zone_output(
zone, run(["firewall-cmd", f"--zone={zone}", "--list-all"], sudo=True)
).get("services", [])
@@ -528,12 +927,36 @@ def set_zone_services(_request: Any, body: dict[str, Any] | None) -> dict[str, A
sudo=True,
)
_reload()
refresh_state(["firewall"])
# Keep the declarative config in sync so the next apply does not
# reconcile the live services back to the stale config value. The
# mutation already applied to live firewalld, so re-stamp the applied
# baseline: cancel-all must revert to this state, not an older snapshot.
cfg = _get_config()
cfg.setdefault("zones", {}).setdefault(zone, {})["services"] = list(services)
stamp_applied(cfg)
_save_config(cfg)
logger.info("Zone '%s' services set to %s", zone, services)
emit_and_refresh("firewall", {"action": "services_set", "zone": zone})
return {"zone": zone, "services": services}
@registry.register(POST_FIREWALL_RICH_RULES_ADD)
def add_rich_rule(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Add rich rule with auto-generated ID from uuid4.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone`` (zone name) and ``rule``
(rich rule string).
Returns:
Dict with ``zone``, ``id`` (generated UUID), and ``rule`` keys.
Raises:
ValueError: If body is missing ``zone`` or ``rule``.
NotFoundError: If the specified zone does not exist.
"""
if not body:
raise ValueError("Request body required")
zone = body.get("zone", "").strip()
@@ -558,13 +981,27 @@ def add_rich_rule(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
rule_id = uuid4().hex[:8]
entry = {"id": rule_id, "rule": rule}
cfg["zones"][zone]["rich_rules"].append(entry)
stamp_applied(cfg)
_save_config(cfg)
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "rich_rule_added", "zone": zone})
return {"zone": zone, "id": rule_id, "rule": rule}
@registry.register(DELETE_FIREWALL_RICH_RULES_REMOVE)
def remove_rich_rule(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Remove rich rule by ID, emit sync event, refresh state.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone`` and ``id`` (rule ID).
Returns:
Dict with ``zone`` and ``id`` keys.
Raises:
ValueError: If body is missing ``zone`` or ``id``.
NotFoundError: If the zone or rule does not exist.
"""
if not body:
raise ValueError("Request body required")
zone = body.get("zone", "").strip()
@@ -596,13 +1033,29 @@ def remove_rich_rule(_request: Any, body: dict[str, Any] | None) -> dict[str, An
zone_cfg["rich_rules"] = [
r for r in zone_cfg.get("rich_rules", []) if r.get("id") != rule_id
]
stamp_applied(cfg)
_save_config(cfg)
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "rich_rule_removed", "zone": zone})
return {"zone": zone, "id": rule_id}
@registry.register(GET_FIREWALL_RICH_RULES)
def list_rich_rules(_request: Any, body: dict[str, Any] | None) -> list[dict[str, Any]]:
"""Return rich rules list for a zone from state store, matched with config IDs.
Rules from the live firewall that match a config entry get their ID
included; rules without a config entry are returned without an ID.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone`` key.
Returns:
List of dicts with ``id`` (if available) and ``rule`` keys.
Raises:
ValueError: If ``zone`` key is missing from body.
"""
if not body or "zone" not in body:
raise ValueError("'zone' is required")
zone = body["zone"]
@@ -623,21 +1076,65 @@ def list_rich_rules(_request: Any, body: dict[str, Any] | None) -> list[dict[str
@registry.register(POST_FIREWALL_MASQUERADE)
def set_masquerade(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Enable/disable masquerade on a zone.
Also syncs the declarative config (and re-stamps the applied baseline)
when the zone exists in the config, so the pending diff and cancel-all
stay consistent with the live zone.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone`` and ``enable`` (boolean).
Returns:
Dict with ``zone`` and ``masquerade`` keys.
Raises:
ValueError: If body is missing ``zone`` or ``enable``, or if
attempting to enable masquerade on the ``public`` zone.
"""
if not body:
raise ValueError("Request body required")
zone = body.get("zone", "").strip()
enable = body.get("enable")
if not zone or enable is None:
raise ValueError("'zone' and 'enable' (bool) are required")
if zone == "public" and enable:
raise ValueError(
"Masquerade (NAT) is not supported on the public zone — enable it on 'internal' or 'vpn' instead"
)
action = "--add-masquerade" if enable else "--remove-masquerade"
run(["firewall-cmd", f"--zone={zone}", action, "--permanent"], sudo=True)
_reload()
refresh_state(["firewall"])
# Keep the declarative config in sync with the live zone so the pending
# diff and the cancel-all baseline stay consistent. Only touch zones that
# already exist in the config — creating a bare zone entry would
# manufacture spurious service/interface diffs on the next poll.
cfg = _get_config()
zone_cfg = cfg.get("zones", {}).get(zone)
if isinstance(zone_cfg, dict):
zone_cfg["masquerade"] = bool(enable)
stamp_applied(cfg)
_save_config(cfg)
emit_and_refresh("firewall", {"action": "masquerade_set", "zone": zone})
return {"zone": zone, "masquerade": bool(enable)}
@registry.register(POST_FIREWALL_FORWARD_PORT_ADD)
def add_forward_port(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Add port forwarding rule with auto-generated ID.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone``, ``port``, ``proto``, and optionally
``toaddr`` and ``toport``.
Returns:
Dict with ``zone``, ``id``, ``port``, and ``proto`` keys.
Raises:
ValueError: If body is missing required fields.
"""
if not body:
raise ValueError("Request body required")
zone = body.get("zone", "").strip()
@@ -667,20 +1164,34 @@ def add_forward_port(_request: Any, body: dict[str, Any] | None) -> dict[str, An
_reload()
fp_id = uuid4().hex[:8]
entry: dict[str, Any] = {"id": fp_id, "port": int(port), "proto": proto}
if toaddr:
if toaddr and toport:
entry["toaddr"] = toaddr
if toport:
entry["toport"] = int(toport)
cfg = _get_config()
cfg.setdefault("zones", {}).setdefault(zone, {}).setdefault("forward_ports", [])
cfg["zones"][zone]["forward_ports"].append(entry)
stamp_applied(cfg)
_save_config(cfg)
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "forward_port_added", "zone": zone})
return {"zone": zone, "id": fp_id, "port": int(port), "proto": proto}
@registry.register(DELETE_FIREWALL_FORWARD_PORT_REMOVE)
def remove_forward_port(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""Remove port forwarding rule.
Args:
_request: The incoming HTTP request (unused).
body: JSON body with ``zone``, ``port``, and ``proto``.
Returns:
Dict with ``zone``, ``port``, and ``proto`` keys.
Raises:
ValueError: If body is missing required fields.
NotFoundError: If the zone or forward port does not exist.
"""
if not body:
raise ValueError("Request body required")
zone = body.get("zone", "").strip()
@@ -721,13 +1232,25 @@ def remove_forward_port(_request: Any, body: dict[str, Any] | None) -> dict[str,
cfg["zones"][zone]["forward_ports"] = [
fp for fp in fps if not (fp.get("port") == port and fp.get("proto") == proto)
]
stamp_applied(cfg)
_save_config(cfg)
refresh_state(["firewall"])
emit_and_refresh("firewall", {"action": "forward_port_removed", "zone": zone})
return {"zone": zone, "port": int(port), "proto": proto}
@registry.register(GET_FIREWALL_STATE)
def get_state(_request: Any, _body: Any) -> dict[str, Any]:
"""Return complete firewall state snapshot.
Args:
_request: The incoming HTTP request (unused).
_body: The request body (unused).
Returns:
Dict with ``active_zones``, ``interfaces``, ``available_services``,
``zones``, ``rich_rules``, and ``timestamp`` keys. Returns empty
dict if state is not populated yet.
"""
fw = _get_state()
if fw is None:
return {}
+55 -5
View File
@@ -10,6 +10,7 @@ import re
from pathlib import Path
from typing import Any
from daemon.handlers.common import emit_and_refresh
from daemon.iface import (
GET_NETWORK_INFER_DHCP_RANGES,
GET_NETWORK_INFER_ZONES,
@@ -21,7 +22,9 @@ from daemon.iface import (
POST_NETWORK_SYSCTL_SET,
)
from daemon.server import NotFoundError, registry
from lib.common import run, validate_interface_name
from lib.common import run, stamp_applied, validate_interface_name
from lib.dnsmasq import get_config as _get_dm_cfg
from lib.dnsmasq import save_config as _save_dm_cfg
from lib.dnsmasq import set_upstreams
from lib.network import (
KNOWN_INTERFACE_KEYS,
@@ -40,16 +43,33 @@ logger = logging.getLogger(__name__)
PROJECT_DIR = Path(__file__).resolve().parent.parent.parent
CONFIG_DIR = PROJECT_DIR / "config" / "network"
DATA_DIR = PROJECT_DIR / "data" / "networkd"
RUNTIME_DIR = Path("/run/vacuum-wall")
_ALLOWED_SYSCTL_KEYS: set[str] = {
"net.ipv4.ip_forward",
"net.ipv4.conf.all.forwarding",
"net.ipv4.conf.all.accept_redirects",
"net.ipv4.conf.default.accept_redirects",
"net.ipv4.conf.all.send_redirects",
"net.ipv4.conf.default.send_redirects",
"net.ipv4.conf.all.rp_filter",
"net.ipv4.icmp_echo_ignore_all",
"net.ipv4.tcp_syncookies",
}
def _copy_and_reload(iface_name: str) -> None:
"""Copy generated 99-<name>.network file to /etc/systemd/network/ and reload."""
validate_interface_name(iface_name)
src = DATA_DIR / f"99-{iface_name}.network"
runtime_src = RUNTIME_DIR / f"99-{iface_name}.network"
dst_dir = Path("/etc/systemd/network")
RUNTIME_DIR.mkdir(exist_ok=True)
runtime_src.write_text(src.read_text())
run(["mkdir", "-p", str(dst_dir)], sudo=True)
dst = dst_dir / f"99-{iface_name}.network"
run(["cp", str(src), str(dst)], sudo=True)
run(["cp", "--", str(runtime_src), str(dst)], sudo=True)
runtime_src.unlink(missing_ok=True)
# Remove lower-priority .network files that match this interface
# (they would override our config due to higher systemd priority)
@@ -107,7 +127,7 @@ def get_interfaces(_request: Any, _body: Any) -> dict[str, Any]:
runtime = parse_networkctl_status(raw)
merged: dict[str, Any] = {}
all_names = set(ifaces_cfg.keys()) | set(runtime.keys()) - {"lo"}
all_names = set(ifaces_cfg.keys()) | set(runtime.keys())
for name in sorted(all_names):
merged[name] = {
"config": ifaces_cfg.get(name, {}),
@@ -194,7 +214,20 @@ def save_interface(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]
)
logger.info("Interface '%s' config saved (applied=%s)", name, deployed)
return {"name": name, "applied": deployed}
# Always stamp the hash so pending-changes detection stays current
# even when deployment fails (e.g. in containerized environments).
# The hash represents the JSON config state, not the system state.
cfg_after = get_config()
stamp_applied(cfg_after)
save_config(cfg_after)
synced = emit_and_refresh(
"networkd", {"action": "interface_saved", "interface": name}
)
return {
"name": name,
"applied": deployed,
"synced": synced,
}
@registry.register(POST_NETWORK_INTERFACE_RELOAD)
@@ -236,9 +269,12 @@ def apply_all(_request: Any, _body: Any) -> dict[str, Any]:
cleaned.append(f)
for f in generated:
tmp = RUNTIME_DIR / f.name
run(["cp", "--", str(f), str(tmp)], sudo=False)
dst = sys_dir / f.name
run(["mkdir", "-p", str(sys_dir)], sudo=True)
run(["cp", str(f), str(dst)], sudo=True)
run(["cp", "--", str(tmp), str(dst)], sudo=True)
tmp.unlink(missing_ok=True)
_full_reload()
@@ -247,10 +283,20 @@ def apply_all(_request: Any, _body: Any) -> dict[str, Any]:
upstreams = collect_upstream_dns(cfg)
if upstreams:
set_upstreams(upstreams)
# Update dnsmasq applied snapshot + hash so pending-changes
# detection stays correct
dm_cfg = _get_dm_cfg()
stamp_applied(dm_cfg)
_save_dm_cfg(dm_cfg)
logger.info("Synced %d DNS upstreams to dnsmasq", len(upstreams))
except Exception:
logger.warning("Failed to sync DNS upstreams to dnsmasq", exc_info=True)
cfg_after = get_config()
stamp_applied(cfg_after)
save_config(cfg_after)
synced = emit_and_refresh("networkd", {"action": "config_applied"})
logger.info(
"Network config applied: %d interfaces, %d stale cleaned",
len(generated),
@@ -260,6 +306,7 @@ def apply_all(_request: Any, _body: Any) -> dict[str, Any]:
"applied": len(generated),
"files": [str(p) for p in generated],
"cleaned": [str(p) for p in cleaned],
"synced": synced,
}
@@ -295,6 +342,8 @@ def set_sysctl(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
raise ValueError("'name' is required")
if not re.match(r"^[a-zA-Z0-9_]+(\.[a-zA-Z0-9_]+)*$", name):
raise ValueError("'name' is not a valid sysctl key")
if name not in _ALLOWED_SYSCTL_KEYS:
raise ValueError("'name' is not a permitted sysctl key")
value = str(body.get("value", "")).strip()
if not value:
raise ValueError("'value' is required")
@@ -311,4 +360,5 @@ def set_sysctl(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
)
logger.info("sysctl %s set to %s", name, value)
emit_and_refresh("networkd", {"action": "sysctl_set", "name": name})
return {"name": name, "value": value}
+416 -174
View File
@@ -2,6 +2,7 @@
import logging
import os
import subprocess
from copy import deepcopy
from pathlib import Path
from typing import Any
@@ -9,11 +10,15 @@ from typing import Any
from jinja2 import Environment, FileSystemLoader
from daemon.iface import (
DELETE_NGINX_BACKENDS_REMOVE,
DELETE_NGINX_DOMAINS_REMOVE,
GET_NGINX_BACKENDS,
GET_NGINX_CONFIG,
GET_NGINX_DOMAINS,
PATCH_NGINX_BACKENDS,
PATCH_NGINX_CONFIG,
POST_NGINX_APPLY,
POST_NGINX_BACKENDS_ADD,
POST_NGINX_CONFIG,
POST_NGINX_DOMAINS_ADD,
POST_NGINX_DOMAINS_UPDATE,
@@ -21,9 +26,21 @@ from daemon.iface import (
POST_NGINX_SSL_APPLY,
POST_NGINX_TEST,
)
from daemon.server import NotFoundError, refresh_state, registry
from daemon.server import ConflictError, NotFoundError, refresh_state, registry
from lib.acme import find_cert_dir
from lib.common import ensure_dirs, load_json, run, run_proc, save_json
from lib.common import (
deep_merge,
ensure_dirs,
load_json,
run,
run_proc,
save_json,
stamp_applied,
strip_apply_meta,
)
from lib.nginx import DEFAULT_SSL, WEBUI_BACKEND
from lib.nginx import _resolve_auth as _ngx_resolve_auth
from lib.nginx import _resolve_paths as _ngx_resolve_paths
logger = logging.getLogger(__name__)
@@ -43,66 +60,76 @@ ENV = Environment(
trim_blocks=True,
)
DEFAULT_SSL: dict[str, Any] = {
"protocols": "TLSv1.2 TLSv1.3",
"ciphers": (
"ECDHE-ECDSA-AES128-GCM-SHA256:"
"ECDHE-RSA-AES128-GCM-SHA256:"
"ECDHE-ECDSA-AES256-GCM-SHA384:"
"ECDHE-RSA-AES256-GCM-SHA384:"
"ECDHE-ECDSA-CHACHA20-POLY1305:"
"ECDHE-RSA-CHACHA20-POLY1305"
),
"prefer_server_ciphers": False,
}
DEFAULT_CONFIG: dict[str, Any] = {
"backends": {},
"domains": {},
"ssl": {**DEFAULT_SSL},
}
def _migrate_config(raw: dict[str, Any]) -> dict[str, Any]:
"""Migrate legacy config formats to the new paths-based model."""
if "management" in raw and raw["management"] is not None:
mgmt = raw["management"]
mgmt_domain = mgmt.get("domain", "")
if mgmt_domain:
domains = raw.setdefault("domains", {})
if mgmt_domain not in domains:
domains[mgmt_domain] = {
"force_ssl": True,
"paths": {},
}
dom = domains[mgmt_domain]
paths = dom.setdefault("paths", {})
if "/" not in paths:
paths["/"] = {
"backend": {
"host": mgmt.get("backend", {}).get("host", "127.0.0.1"),
"port": mgmt.get("backend", {}).get("port", 9090),
"proto": "http",
},
"is_management": True,
}
if mgmt.get("auth"):
paths["/"]["auth"] = mgmt["auth"]
if "/ws" not in paths:
paths["/ws"] = {
"backend": {"host": "127.0.0.1", "port": 9091, "proto": "http"},
"is_websocket": True,
}
del raw["management"]
# ---------------------------------------------------------------------------
# Migration
def _migrate_config(raw: dict[str, Any]) -> tuple[dict[str, Any], bool]:
"""Migrate legacy config to backends model. Returns (config, changed)."""
c1 = _ensure_webui_backend(raw)
c2 = _migrate_mgmt_domains(raw)
return raw, c1 or c2
def _ensure_webui_backend(raw: dict[str, Any]) -> bool:
"""Create builtin webui backend if not yet migrated. Returns True if changed."""
backends = raw.setdefault("backends", {})
webui = backends.get("webui")
if webui and webui.get("_migrated"):
return False
backends["webui"] = deepcopy(WEBUI_BACKEND)
backends["webui"]["_migrated"] = True
# Harvest auth from legacy path-level auth if present
for dom in raw.get("domains", {}).values():
if "paths" not in dom and "backend" in dom:
dom["paths"] = {
"/": {
"backend": dom.pop("backend"),
"headers": dom.pop("headers", {}),
}
}
return raw
paths = dom.get("paths", {})
root = paths.get("/", {})
if root.get("auth"):
backends["webui"]["auth"] = root["auth"]
break
return True
def _migrate_mgmt_domains(raw: dict[str, Any]) -> bool:
"""Migrate legacy mgmt domains to backend refs. Returns True if anything changed."""
backends = raw.get("backends", {})
if not backends.get("webui", {}).get("_migrated"):
return False
domains = raw.setdefault("domains", {})
changed = False
for _name, dom in list(domains.items()):
if dom.get("backend") == "webui":
continue
if dom.get("application") == "webui":
del dom["application"]
changed = True
paths = dom.get("paths", {})
root = paths.get("/", {})
ws = paths.get("/ws", {})
root_backend = root.get("backend", {})
ws_backend = ws.get("backend", {})
is_mgmt_root = root.get("is_management") or (
root_backend.get("host") == "127.0.0.1" and root_backend.get("port") == 9090
)
is_mgmt_ws = ws.get("is_websocket") or (
ws_backend.get("host") == "127.0.0.1" and ws_backend.get("port") == 9091
)
if is_mgmt_root and is_mgmt_ws:
dom["backend"] = "webui"
dom.pop("paths", None)
dom.pop("auth", None)
changed = True
return changed
# ---------------------------------------------------------------------------
# Config helpers
def _get_state() -> dict[str, Any] | None:
@@ -120,9 +147,13 @@ def _get_config() -> dict[str, Any]:
raw = deepcopy(DEFAULT_CONFIG)
if "ssl" not in raw:
raw["ssl"] = deepcopy(DEFAULT_SSL)
raw = _migrate_config(raw)
_save_config(raw)
return raw
cfg, changed = _migrate_config(raw)
if changed:
_save_config(cfg)
# The on-disk config format changed under the state store's feet;
# re-collect so cached state (e.g. the domains list) matches the file.
refresh_state(["nginx"])
return cfg
def _save_config(cfg: dict[str, Any]) -> None:
@@ -130,14 +161,122 @@ def _save_config(cfg: dict[str, Any]) -> None:
save_json(CONFIG_FILE, cfg)
def _generate_server_conf(domain_cfg: dict[str, Any]) -> str:
# ---------------------------------------------------------------------------
# Backend CRUD
def _get_backends() -> dict[str, Any]:
"""Return the backends dict from config, creating builtin webui if needed."""
cfg = _get_config()
backends = cfg.get("backends", {})
webui = backends.get("webui")
if not webui or not webui.get("_migrated"):
backends["webui"] = deepcopy(WEBUI_BACKEND)
backends["webui"]["_migrated"] = True
cfg["backends"] = backends
_save_config(cfg)
# Config file was rewritten (builtin backend materialized); re-collect
# so cached state matches the file.
refresh_state(["nginx"])
return backends
def _validate_paths(paths: dict) -> None:
"""Validate each path has backend with host, port, proto."""
for path_str, path_cfg in paths.items():
backend = path_cfg.get("backend")
if not backend:
raise ValueError(f"path {path_str!r} missing 'backend'")
if not isinstance(backend, dict):
raise ValueError(f"path {path_str!r} 'backend' must be a dict")
if not backend.get("host"):
raise ValueError(f"path {path_str!r} 'backend' missing 'host'")
if not backend.get("port"):
raise ValueError(f"path {path_str!r} 'backend' missing 'port'")
if not backend.get("proto"):
raise ValueError(f"path {path_str!r} 'backend' missing 'proto'")
def _add_backend(
name: str, label: str, paths: dict, auth: dict | None = None, builtin: bool = False
) -> None:
"""Add a backend. Validate name uniqueness and path schema."""
_validate_paths(paths)
cfg = _get_config()
backends = cfg.setdefault("backends", {})
if name in backends:
raise ValueError(f"Backend {name!r} already exists")
entry: dict[str, Any] = {"label": label, "paths": paths}
if builtin:
entry["builtin"] = True
if auth is not None:
if "htpasswd" in auth:
htpasswd_path = Path(auth["htpasswd"])
if not htpasswd_path.is_absolute():
auth = {**auth, "htpasswd": str(PROJECT_DIR / htpasswd_path)}
entry["auth"] = auth
backends[name] = entry
_save_config(cfg)
def _update_backend(
name: str,
label: str | None = None,
paths: dict | None = None,
auth: dict | None | bool = None,
) -> None:
"""Update a backend. Cannot edit builtin backends. auth=False removes auth."""
cfg = _get_config()
backends = cfg.setdefault("backends", {})
if name not in backends:
raise KeyError(name)
if backends[name].get("builtin"):
raise ValueError("Cannot modify builtin backend")
entry = backends[name]
if label is not None:
entry["label"] = label
if paths is not None:
_validate_paths(paths)
entry["paths"] = paths
if auth is False or auth is None:
entry.pop("auth", None)
elif isinstance(auth, dict):
if "htpasswd" in auth:
htpasswd_path = Path(auth["htpasswd"])
if not htpasswd_path.is_absolute():
auth = {**auth, "htpasswd": str(PROJECT_DIR / htpasswd_path)}
entry["auth"] = auth
_save_config(cfg)
def _remove_backend(name: str) -> None:
"""Remove a non-builtin backend. Raise ConflictError if domains reference it."""
cfg = _get_config()
backends = cfg.setdefault("backends", {})
if name not in backends:
raise KeyError(name)
if backends[name].get("builtin"):
raise ValueError("Cannot remove builtin backend")
for dom_name, dom in cfg.get("domains", {}).items():
if dom.get("backend") == name:
raise ConflictError(
f"Backend {name!r} is referenced by domain {dom_name!r}"
)
del backends[name]
_save_config(cfg)
# ---------------------------------------------------------------------------
# Site generation
def _generate_server_conf(domain_cfg: dict[str, Any], backends: dict[str, Any]) -> str:
"""Render an nginx server block config from a domain entry via Jinja."""
tmpl = ENV.get_template("nginx/server_block.conf")
acme_home_path = PROJECT_DIR / "data" / "acme"
acme_cert_dir = str(find_cert_dir(domain_cfg["domain"], acme_home_path))
paths = domain_cfg.get("paths", {})
paths = _ngx_resolve_paths(domain_cfg, backends)
has_management = any(p.get("is_management") for p in paths.values())
# Resolve custom cert paths for cert=="file"
cert_cfg = domain_cfg.get("cert")
if isinstance(cert_cfg, dict):
cert_path = cert_cfg.get("cert_path", "")
@@ -152,8 +291,9 @@ def _generate_server_conf(domain_cfg: dict[str, Any]) -> str:
cert=domain_cfg.get("cert"),
cert_path=cert_path,
cert_key_path=cert_key_path,
domain_auth=domain_cfg.get("auth"),
domain_auth=_ngx_resolve_auth(domain_cfg, backends),
has_management=has_management,
static_root=str(PROJECT_DIR / "webui" / "static"),
acme_cert_dir=acme_cert_dir,
certs_dir=str(PROJECT_DIR / "data" / "certs"),
acme_webroot=str(PROJECT_DIR / "data" / "acme" / "www"),
@@ -176,11 +316,11 @@ def _write_include_file() -> None:
"""Write the system include file that references all per-site configs."""
tmpl = ENV.get_template("nginx/include.conf")
content = tmpl.render(sites_glob=str(SITES_DIR / "*.conf"))
tmp = Path("/tmp") / "vacuum-wall-include.tmp"
with open(tmp, "w") as f:
f.write(content)
tmp = Path("/run/vacuum-wall/include.tmp")
tmp.parent.mkdir(exist_ok=True)
tmp.write_text(content)
os.chmod(tmp, 0o644)
run(["cp", str(tmp), str(INCLUDE_FILE)], sudo=True)
run(["cp", "--", str(tmp), str(INCLUDE_FILE)], sudo=True)
run(["chown", "root:root", str(INCLUDE_FILE)], sudo=True)
tmp.unlink(missing_ok=True)
@@ -194,11 +334,11 @@ def _write_ssl_snippet() -> None:
ssl_cfg.setdefault("ciphers", DEFAULT_SSL["ciphers"])
tmpl = ENV.get_template("nginx/ssl_snippet.conf")
content = tmpl.render(ssl=ssl_cfg)
tmp = Path("/tmp") / "vacuum-wall-ssl-snippet.tmp"
with open(tmp, "w") as f:
f.write(content)
tmp = Path("/run/vacuum-wall/ssl-snippet.tmp")
tmp.parent.mkdir(exist_ok=True)
tmp.write_text(content)
os.chmod(tmp, 0o644)
run(["cp", str(tmp), str(SSL_SNIPPET)], sudo=True)
run(["cp", "--", str(tmp), str(SSL_SNIPPET)], sudo=True)
run(["chown", "root:root", str(SSL_SNIPPET)], sudo=True)
tmp.unlink(missing_ok=True)
@@ -222,15 +362,59 @@ def _reload_nginx() -> None:
logger.info("nginx configuration applied and reloaded")
def _ensure_self_signed_cert(domain: str) -> None:
"""Auto-generate a self-signed cert for *domain* if not yet present."""
certs_dir = PROJECT_DIR / "data" / "certs"
certs_dir.mkdir(parents=True, exist_ok=True)
cert_file = certs_dir / f"{domain}.crt"
key_file = certs_dir / f"{domain}.key"
if cert_file.is_file() and key_file.is_file():
return
logger.info("Auto-generating self-signed cert for %s", domain)
subprocess.run(
[
"openssl",
"req",
"-x509",
"-newkey",
"rsa:2048",
"-keyout",
str(key_file),
"-out",
str(cert_file),
"-days",
"365",
"-nodes",
"-subj",
f"/CN={domain}",
],
capture_output=True,
text=True,
check=True,
timeout=30,
)
cert_file.chmod(0o644)
key_file.chmod(0o600)
def _write_all_sites() -> None:
"""Regenerate all site configs and ACME challenge site."""
ensure_dirs(SITES_DIR)
cfg = _get_config()
backends = cfg.get("backends", {})
# Auto-generate self-signed certs for management domains that need them
for name, dom in cfg.get("domains", {}).items():
cert = dom.get("cert")
paths = _ngx_resolve_paths(dom, backends)
has_management = any(p.get("is_management") for p in paths.values())
if has_management and (cert == "selfsigned" or cert is None):
_ensure_self_signed_cert(name)
existing = set(SITES_DIR.iterdir()) if SITES_DIR.exists() else set()
written: set[str] = set()
for name, dom in cfg.get("domains", {}).items():
dom_copy = dict(dom, domain=name)
conf = _generate_server_conf(dom_copy)
conf = _generate_server_conf(dom_copy, backends)
_write_site(name, conf)
written.add(f"{name}.conf")
@@ -252,6 +436,10 @@ def _write_all_sites() -> None:
os.replace(tmp, site)
# ---------------------------------------------------------------------------
# Auth helpers
def _hash_password(password: str) -> str:
"""Hash *password* using SHA-256 crypt via passlib."""
from passlib.hash import sha256_crypt
@@ -259,13 +447,19 @@ def _hash_password(password: str) -> str:
return sha256_crypt.hash(password)
def _write_htpasswd(user: str, password: str) -> None:
"""Add or update a user entry in the .htpasswd file using SHA-256 hashing."""
ensure_dirs(DATA_DIR)
def _write_htpasswd(
user: str, password: str, htpasswd_path: Path | None = None
) -> None:
"""Add or update a user entry in the .htpasswd file using SHA-256 hashing.
If *htpasswd_path* is not given, defaults to :data:`HTPASSWD_FILE`.
"""
target = htpasswd_path or HTPASSWD_FILE
ensure_dirs(target.parent)
hashed = _hash_password(password)
existing: dict[str, str] = {}
if HTPASSWD_FILE.exists():
with open(HTPASSWD_FILE) as f:
if target.exists():
with open(target) as f:
for line in f:
line = line.strip()
if not line or line.startswith("#"):
@@ -274,12 +468,12 @@ def _write_htpasswd(user: str, password: str) -> None:
if len(parts) == 2:
existing[parts[0]] = line
existing[user] = f"{user}:{hashed}"
tmp = HTPASSWD_FILE.with_suffix(".tmp")
tmp = target.with_suffix(".tmp")
with open(tmp, "w") as f:
for _uname, entry in existing.items():
f.write(entry + "\n")
os.chmod(tmp, 0o640)
os.replace(tmp, HTPASSWD_FILE)
os.replace(tmp, target)
def _get_nginx_state() -> dict[str, Any]:
@@ -300,7 +494,8 @@ def get_config(_request: Any, _body: Any) -> dict[str, Any]:
ng = _get_nginx_state()
if ng:
return ng.get("config", {})
return _get_config()
cfg = _get_config()
return strip_apply_meta(cfg)
@registry.register(POST_NGINX_CONFIG)
@@ -318,8 +513,6 @@ def patch_config(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""PATCH /nginx/config — deep-merge partial updates into current config."""
if not body:
raise ValueError("Request body required")
from lib.common import deep_merge
current = _get_config()
merged = deep_merge(current, body)
_save_config(merged)
@@ -338,10 +531,7 @@ def get_domains(_request: Any, _body: Any) -> list[dict[str, Any]]:
@registry.register(POST_NGINX_DOMAINS_ADD)
def add_domain(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /nginx/domains/add — add a new reverse-proxy domain entry.
Accepts either legacy backend_* fields or a ``paths`` map.
"""
"""POST /nginx/domains/add — add a new reverse-proxy domain entry."""
if not body:
raise ValueError("Request body required")
domain = body.get("domain", "").strip()
@@ -351,70 +541,36 @@ def add_domain(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
if domain in cfg["domains"]:
raise ValueError(f"Domain {domain!r} already configured")
paths = body.get("paths")
backend_name = body.get("backend", "").strip()
if not backend_name:
raise ValueError("'backend' is required")
if backend_name not in cfg.get("backends", {}):
raise ValueError(f"Backend {backend_name!r} not found")
cert = body.get("cert")
force_ssl = body.get("force_ssl", True)
if paths is not None:
entry: dict[str, Any] = {
"paths": paths,
"force_ssl": force_ssl,
}
if cert is not None:
entry["cert"] = cert
else:
backend_host = body.get("backend_host", "").strip()
backend_port = body.get("backend_port")
backend_proto = body.get("backend_proto", "http").strip() or "http"
extra_headers = body.get("extra_headers")
if not backend_host:
raise ValueError("'backend_host' is required")
if backend_port is None:
raise ValueError("'backend_port' is required")
entry = {
"paths": {
"/": {
"backend": {
"host": backend_host,
"port": int(backend_port),
"proto": backend_proto,
},
"headers": extra_headers or {},
}
},
"force_ssl": force_ssl,
}
if cert is not None:
entry["cert"] = cert
entry: dict[str, Any] = {
"backend": backend_name,
"force_ssl": force_ssl,
}
if cert is not None:
entry["cert"] = cert
# Handle auth credentials for management domain
auth_user = body.get("auth_user", "").strip()
auth_pass = body.get("auth_pass", "")
if auth_user and auth_pass:
_write_htpasswd(auth_user, auth_pass)
auth_dict = {"user": auth_user, "htpasswd": str(HTPASSWD_FILE)}
paths_entry = entry.get("paths", {})
for _ppath, pcfg in paths_entry.items():
if pcfg.get("is_management"):
pcfg["auth"] = auth_dict
break
entry["auth"] = auth_dict
# Handle auth credentials for management paths
auth_user = body.get("auth_user", "").strip()
auth_pass = body.get("auth_pass", "").strip()
if auth_user and auth_pass:
_write_htpasswd(auth_user, auth_pass)
auth_entry = {
"user": auth_user,
"htpasswd": str(HTPASSWD_FILE),
}
# Store auth on root path if it exists
root_path = entry.get("paths", {}).get("/")
if root_path:
root_path["auth"] = auth_entry
# Also store at domain level for template
entry["auth"] = auth_entry
auth = body.get("auth")
if auth is not None:
# Write htpasswd file when a password is provided, then store
# only {user, htpasswd path} — never persist the raw password.
if auth.get("user") and auth.get("pass"):
htpasswd_path = auth.get(
"htpasswd", str(PROJECT_DIR / "data" / "nginx" / ".htpasswd")
)
if isinstance(htpasswd_path, str) and not Path(htpasswd_path).is_absolute():
htpasswd_path = PROJECT_DIR / htpasswd_path
_write_htpasswd(auth["user"], auth["pass"], Path(htpasswd_path))
entry["auth"] = {"user": auth["user"], "htpasswd": str(htpasswd_path)}
else:
entry["auth"] = auth
cfg["domains"][domain] = entry
_save_config(cfg)
@@ -455,43 +611,126 @@ def update_domain(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
raise NotFoundError(f"Domain {domain!r} not configured")
entry = cfg["domains"][domain]
# Path removal: if body has `path` key (string) but no `paths`/`backend`/`headers`
path_to_remove = body.get("path")
if path_to_remove is not None and "paths" not in body and "backend" not in body and "headers" not in body:
paths = entry.get("paths", {})
if path_to_remove in paths:
del paths[path_to_remove]
if not paths:
entry.pop("paths", None)
_save_config(cfg)
refresh_state(["nginx"])
return {"domain": domain, "path_removed": path_to_remove}
new_backend = body.get("backend")
if new_backend:
new_backend = new_backend.strip()
if new_backend not in cfg.get("backends", {}):
raise ValueError(f"Backend {new_backend!r} not found")
entry["backend"] = new_backend
updates = {k: v for k, v in body.items() if k not in ("domain", "path")}
if "paths" in updates:
entry["paths"] = updates["paths"]
else:
paths = entry.setdefault("paths", {})
if "backend" in updates:
root = paths.setdefault("/", {"backend": {}, "headers": {}})
root["backend"] = updates["backend"]
if "headers" in updates:
root = paths.setdefault("/", {"backend": {}, "headers": {}})
root["headers"] = updates["headers"]
for key, val in updates.items():
if key in ("backend", "headers", "paths"):
continue
if isinstance(val, dict) and key in entry:
entry[key].update(val)
if "cert" in body:
if body["cert"] is None:
entry.pop("cert", None)
else:
entry[key] = val
entry["cert"] = body["cert"]
if "force_ssl" in body:
entry["force_ssl"] = body["force_ssl"]
if "auth" in body:
if body["auth"] is None:
entry.pop("auth", None)
else:
# Normalize: if auth has `pass`, write htpasswd and store only
# `{user, htpasswd path}` — never persist the raw password.
if body["auth"].get("user") and body["auth"].get("pass"):
htpasswd_path = body["auth"].get(
"htpasswd", str(PROJECT_DIR / "data" / "nginx" / ".htpasswd")
)
if (
isinstance(htpasswd_path, str)
and not Path(htpasswd_path).is_absolute()
):
htpasswd_path = PROJECT_DIR / htpasswd_path
_write_htpasswd(
body["auth"]["user"], body["auth"]["pass"], Path(htpasswd_path)
)
entry["auth"] = {
"user": body["auth"]["user"],
"htpasswd": str(htpasswd_path),
}
else:
entry["auth"] = body["auth"]
_save_config(cfg)
refresh_state(["nginx"])
return {"domain": domain}
@registry.register(GET_NGINX_BACKENDS)
def get_backends(_request: Any, _body: Any) -> dict[str, Any]:
"""GET /nginx/backends — return backends with secrets stripped."""
backends = _get_backends()
result: dict[str, Any] = {}
for name, be in backends.items():
entry = deepcopy(be)
entry.pop("_migrated", None)
if "auth" in entry:
entry["has_auth"] = entry["auth"] is not None
del entry["auth"]
else:
entry["has_auth"] = False
result[name] = entry
return result
@registry.register(PATCH_NGINX_BACKENDS)
def patch_backends(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""PATCH /nginx/backends — deep-merge partial updates into a backend entry."""
if not body:
raise ValueError("Request body required")
name = body.get("name", "").strip()
if not name:
raise ValueError("'name' is required")
cfg = _get_config()
backends = cfg.setdefault("backends", {})
if name not in backends:
raise KeyError(name)
if backends[name].get("builtin"):
raise ValueError("Cannot modify builtin backend")
update_data = {k: v for k, v in body.items() if k != "name"}
if "auth" in update_data and (
update_data["auth"] is False or update_data["auth"] is None
):
backends[name].pop("auth", None)
update_data.pop("auth")
backends[name] = deep_merge(backends[name], update_data)
_save_config(cfg)
refresh_state(["nginx"])
return {"backend": name}
@registry.register(POST_NGINX_BACKENDS_ADD)
def add_backend(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /nginx/backends/add — add a new backend."""
if not body:
raise ValueError("Request body required")
name = body.get("name", "").strip()
label = body.get("label", "").strip()
paths = body.get("paths")
auth = body.get("auth")
if not name:
raise ValueError("'name' is required")
if not label:
raise ValueError("'label' is required")
if not paths:
raise ValueError("'paths' is required")
_add_backend(name, label, paths, auth=auth if auth else None)
refresh_state(["nginx"])
return {"backend": name}
@registry.register(DELETE_NGINX_BACKENDS_REMOVE)
def remove_backend(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""DELETE /nginx/backends/remove — remove a non-builtin backend."""
if not body:
raise ValueError("Request body required")
name = body.get("name", "").strip()
if not name:
raise ValueError("'name' is required")
_remove_backend(name)
refresh_state(["nginx"])
return {"backend": name}
@registry.register(POST_NGINX_APPLY)
def apply(_request: Any, _body: Any) -> dict[str, Any]:
"""POST /nginx/apply — render all configs, test, and reload nginx."""
@@ -502,6 +741,9 @@ def apply(_request: Any, _body: Any) -> dict[str, Any]:
if not ok:
raise RuntimeError(f"nginx config test failed: {msg}")
_reload_nginx()
cfg_after = _get_config()
stamp_applied(cfg_after)
_save_config(cfg_after)
refresh_state(["nginx"])
return {"applied": True}
+254
View File
@@ -0,0 +1,254 @@
"""Aggregate status handler.
Exposes pending changes across all subsystems, a single apply-all
endpoint that invokes each subsystem's apply in the correct order, and a
cancel-all endpoint that reverts pending edits to the last applied config.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any
from daemon.handlers import dnsmasq as _dnsmasq_h
from daemon.handlers import firewall as _firewall_h
from daemon.handlers import nginx as _nginx_h
from daemon.handlers.dnsmasq import apply_config as dnsmasq_apply_config
from daemon.handlers.firewall import config_apply as firewall_config_apply
from daemon.handlers.network import apply_all as network_apply_all
from daemon.handlers.nginx import apply as nginx_apply
from daemon.handlers.wireguard import apply as wireguard_apply
from daemon.iface import (
GET_STATUS_PENDING,
POST_STATUS_APPLY_ALL,
POST_STATUS_CANCEL_ALL,
)
from daemon.server import refresh_state, registry
from lib import network as _net
from lib import wireguard as _wg
from lib.common import revert_to_applied
from lib.firewall import fw_change_summary
from lib.state import state as state_store
logger = logging.getLogger(__name__)
SYS_ORDER = ["networkd", "firewall", "wireguard", "dnsmasq", "nginx"]
SYS_LABELS = {
"networkd": "Network",
"firewall": "Firewall",
"wireguard": "WireGuard",
"dnsmasq": "DHCP/DNS",
"nginx": "Nginx",
}
SYS_APPLY = {
"networkd": network_apply_all,
"firewall": firewall_config_apply,
"wireguard": wireguard_apply,
"dnsmasq": dnsmasq_apply_config,
"nginx": nginx_apply,
}
# (module, attribute) pairs for each subsystem's on-disk config path.
# Resolved at call time so tests can monkeypatch the module constants.
SYS_CONFIG_PATHS: dict[str, tuple[Any, str]] = {
"firewall": (_firewall_h, "CONFIG_FILE"),
"dnsmasq": (_dnsmasq_h, "CONFIG_PATH"),
"nginx": (_nginx_h, "CONFIG_FILE"),
"wireguard": (_wg, "CONFIG_PATH"),
"networkd": (_net, "CONFIG_FILE"),
}
@registry.register(GET_STATUS_PENDING)
def status_pending(_request: Any, _body: Any) -> dict[str, Any]:
"""Aggregate pending changes across all subsystems.
Returns:
Dict with per-subsystem pending status and total change count.
The firewall section also carries advisory `uncovered_interfaces`
and `coverage_warnings` fields (network-config interfaces not in
any live zone); they are never counted in `needs_apply`,
`change_count`, or `total_changes`.
"""
fw = state_store.get("firewall") or {}
pending_fw = fw.get("pending", {})
fw_needs_apply = pending_fw.get("needs_apply", False)
fw_pending_list = pending_fw.get("pending", [])
fw_changes = []
for c in fw_pending_list:
zone = c.get("zone", "unknown")
ctype = c.get("type", "unknown")
summary = fw_change_summary(zone, ctype, c)
fw_changes.append({"summary": summary, "detail": ""})
uncovered = fw.get("uncovered_interfaces") or []
coverage_warnings = (
[
"Interfaces not in any firewall zone: "
f"{', '.join(uncovered)} — clients on those segments lose "
"connectivity and DHCP"
]
if uncovered
else []
)
fw_result = {
"needs_apply": fw_needs_apply,
"change_count": len(fw_changes),
"changes": fw_changes,
"uncovered_interfaces": uncovered,
"coverage_warnings": coverage_warnings,
}
hash_subsystems = {
"dnsmasq": _hash_subsystem("dnsmasq", state_store.get("dnsmasq")),
"nginx": _hash_subsystem("nginx", state_store.get("nginx")),
"wireguard": _hash_subsystem("wireguard", state_store.get("wireguard")),
"networkd": _hash_subsystem("networkd", state_store.get("networkd")),
}
total = len(fw_changes)
for _name, result in hash_subsystems.items():
total += len(result["changes"])
return {
"firewall": fw_result,
"dnsmasq": hash_subsystems["dnsmasq"],
"nginx": hash_subsystems["nginx"],
"wireguard": hash_subsystems["wireguard"],
"networkd": hash_subsystems["networkd"],
"total_changes": total,
}
@registry.register(POST_STATUS_APPLY_ALL)
def status_apply_all(_request: Any, _body: Any) -> dict[str, Any]:
"""Apply pending changes for all subsystems in dependency order.
Order: network -> firewall -> wireguard -> dnsmasq -> nginx.
Args:
_request: The incoming HTTP request (unused).
_body: Optional JSON body; ``{"force": true}`` is forwarded to the
firewall apply, overriding its management-lockout and
interface-coverage guards. Other subsystems ignore it.
Returns:
Dict with applied subsystems and any errors encountered.
"""
applied = []
errors = {}
force = bool(_body and _body.get("force"))
pending_data = status_pending(None, None)
fw_pending = pending_data["firewall"]["needs_apply"]
hash_pending = {
"dnsmasq": pending_data["dnsmasq"]["pending_changes"],
"nginx": pending_data["nginx"]["pending_changes"],
"wireguard": pending_data["wireguard"]["pending_changes"],
"networkd": pending_data["networkd"]["pending_changes"],
}
for name in SYS_ORDER:
if name == "firewall":
if not fw_pending:
continue
else:
if not hash_pending.get(name, False):
continue
handler = SYS_APPLY[name]
try:
# Only the firewall apply honors `force` (its lockout and
# coverage guards); forward it there, not to other subsystems.
body = {"force": True} if (name == "firewall" and force) else None
handler(None, body)
applied.append(name)
except Exception as exc:
label = SYS_LABELS.get(name, name)
errors[label] = str(exc)
logger.error("Apply-all failed for %s: %s", name, exc)
refresh_state(SYS_ORDER)
return {"applied": applied, "errors": errors}
def _config_path(name: str) -> Path:
"""Return the on-disk config path for subsystem *name*.
Resolved via the owning module at call time so tests can monkeypatch
the module constants (e.g. ``lib.wireguard.CONFIG_PATH``).
"""
module, attr = SYS_CONFIG_PATHS[name]
return getattr(module, attr)
@registry.register(POST_STATUS_CANCEL_ALL)
def status_cancel_all(_request: Any, _body: Any) -> dict[str, Any]:
"""Revert pending changes for all subsystems to the last applied config.
Restores each pending subsystem's config file from its recorded
``_last_applied_config`` snapshot, discarding unapplied edits.
Subsystems without a recorded baseline (never applied) are skipped
with a reason instead of being reset. No live-system commands run —
cancel only touches the declarative config files.
Returns:
Dict with ``cancelled`` (list of reverted subsystems),
``skipped`` (label -> reason), and ``errors`` (label -> message).
"""
pending_data = status_pending(None, None)
fw_pending = pending_data["firewall"]["needs_apply"]
hash_pending = {
"dnsmasq": pending_data["dnsmasq"]["pending_changes"],
"nginx": pending_data["nginx"]["pending_changes"],
"wireguard": pending_data["wireguard"]["pending_changes"],
"networkd": pending_data["networkd"]["pending_changes"],
}
cancelled: list[str] = []
skipped: dict[str, str] = {}
errors: dict[str, str] = {}
for name in SYS_ORDER:
pending = fw_pending if name == "firewall" else hash_pending.get(name, False)
if not pending:
continue
label = SYS_LABELS.get(name, name)
try:
ok, reason = revert_to_applied(_config_path(name))
if ok:
cancelled.append(name)
logger.info("Cancelled pending changes for %s", name)
else:
skipped[label] = reason
logger.warning("Cancel-all skipped %s: %s", label, reason)
except Exception as exc:
errors[label] = str(exc)
logger.error("Cancel-all failed for %s: %s", name, exc)
if cancelled:
refresh_state(SYS_ORDER)
return {"cancelled": cancelled, "skipped": skipped, "errors": errors}
def _hash_subsystem(name: str, state: dict[str, Any] | None) -> dict[str, Any]:
"""Build pending result for a hash-based subsystem."""
if state is None:
return {"pending_changes": False, "summary": "Up to date", "changes": []}
status = state.get("status", {})
pending = status.get("pending_changes", False)
label = SYS_LABELS.get(name, name)
if pending:
summary = f"{label} configuration has unapplied changes"
return {
"pending_changes": True,
"summary": summary,
"changes": [{"summary": summary, "detail": ""}],
}
return {"pending_changes": False, "summary": "Up to date", "changes": []}
+35
View File
@@ -0,0 +1,35 @@
"""System metrics handler.
Returns CPU load, memory usage, and per-interface network traffic stats.
"""
from __future__ import annotations
import logging
from typing import Any
from daemon.iface import GET_SYSTEM_METRICS
from daemon.server import registry
from lib.state import state as state_store
logger = logging.getLogger(__name__)
@registry.register(GET_SYSTEM_METRICS)
def system_metrics(_request: Any, _body: Any) -> dict[str, Any]:
"""Return system-wide metrics.
Reads from pre-collected state (CPU load, memory, network traffic).
Returns:
Dict with load, memory, swap, and traffic data.
"""
sys_state = state_store.get("system")
if sys_state is None:
return {
"load": {"load1": 0.0, "load5": 0.0, "load15": 0.0},
"memory": {"total": 0, "available": 0, "used": 0, "used_pct": 0},
"swap": {"total": 0, "used": 0, "used_pct": 0},
"traffic": {},
}
return sys_state
+459 -149
View File
@@ -2,92 +2,68 @@
import logging
import os
from copy import deepcopy
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from jinja2 import Environment, FileSystemLoader
from daemon.handlers.common import emit_and_refresh
from daemon.iface import (
DELETE_WIREGUARD_CLASSES,
DELETE_WIREGUARD_CLASSES_DOWN,
DELETE_WIREGUARD_PEERS_REMOVE,
GET_WIREGUARD_CLASS_STATUS,
GET_WIREGUARD_CLASSES,
GET_WIREGUARD_CONFIG,
GET_WIREGUARD_PEER_STATUS,
GET_WIREGUARD_PEERS,
GET_WIREGUARD_STATUS,
PATCH_WIREGUARD_CLASSES,
PATCH_WIREGUARD_CONFIG,
POST_WIREGUARD_APPLY,
POST_WIREGUARD_CLASS_INIT_KEYS,
POST_WIREGUARD_CLASSES,
POST_WIREGUARD_CLASSES_UP,
POST_WIREGUARD_CONFIG,
POST_WIREGUARD_DOWN,
POST_WIREGUARD_GENERATE_CLIENT,
POST_WIREGUARD_INITIALIZE,
POST_WIREGUARD_PEERS_ADD,
)
from daemon.server import NotFoundError, refresh_state, registry
from lib.common import deep_merge, load_json, run, run_proc, save_json
from daemon.server import ConflictError, NotFoundError, registry
from lib.common import deep_merge, run, stamp_applied, strip_apply_meta
from lib.wireguard import (
_class_interface_name,
_class_peers,
_ensure_access_classes,
_wg_conf_path,
generate_class_conf,
generate_class_keypair,
generate_conf,
generate_keypair,
)
from lib.wireguard import (
generate_client_conf as _gen_client_conf,
)
from lib.wireguard import (
get_config as _get_wireguard_config,
)
from lib.wireguard import (
get_peers as _get_wireguard_peers,
)
from lib.wireguard import (
save_config as _save_wireguard_config,
)
logger = logging.getLogger(__name__)
PROJECT_DIR = Path(__file__).resolve().parent.parent.parent
CONFIG_PATH = PROJECT_DIR / "config" / "wireguard" / "config.json"
WG_CONF_PATH = "/etc/wireguard/wg0.conf"
WG_QUICK_BIN = "wg-quick"
WG_BIN = "wg"
ENV = Environment(
loader=FileSystemLoader(str(PROJECT_DIR / "system")),
autoescape=False,
lstrip_blocks=True,
trim_blocks=True,
)
DEFAULT_CONFIG: dict[str, Any] = {
"interface": {
"name": "wg0",
"listen_port": 51820,
"private_key": "",
"public_key": "",
"addresses": ["10.137.0.1/24"],
"post_up": None,
"post_down": None,
},
"peers": {},
}
def _get_state() -> dict[str, Any] | None:
"""Retrieve cached WireGuard state from the global state store."""
from lib.state import state as state_store
return state_store.get("wireguard")
def _get_config() -> dict[str, Any]:
"""Load and merge the WireGuard config with defaults."""
return deep_merge(deepcopy(DEFAULT_CONFIG), load_json(CONFIG_PATH))
def _save_config(cfg: dict[str, Any]) -> None:
"""Persist the WireGuard config to disk."""
save_json(CONFIG_PATH, cfg)
def _generate_conf(cfg: dict[str, Any]) -> str:
"""Render the WireGuard server config file from Jinja template."""
tmpl = ENV.get_template("wireguard.conf")
return tmpl.render(
timestamp=datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
interface=cfg["interface"],
peers=cfg.get("peers", {}),
)
def _get_wg_state() -> dict[str, Any]:
"""Return cached WireGuard state, or empty dict if not yet loaded."""
wg = _get_state()
if wg is None:
return {}
return wg
from lib.state import state as state_store
wg = state_store.get("wireguard")
return {} if wg is None else wg
# ---------------------------------------------------------------------------
@@ -100,33 +76,53 @@ def get_config(_request: Any, _body: Any) -> dict[str, Any]:
wg = _get_wg_state()
if wg:
return wg.get("config", {})
cfg = _get_config()
safe = dict(cfg)
cfg = _get_wireguard_config()
safe = strip_apply_meta(cfg)
if "interface" in safe:
safe["interface"] = dict(safe["interface"])
safe["interface"].pop("private_key", None)
if "access_classes" in safe:
for ck, cv in safe["access_classes"].items():
if isinstance(cv, dict):
safe["access_classes"][ck] = dict(cv)
safe["access_classes"][ck].pop("private_key", None)
return safe
@registry.register(POST_WIREGUARD_CONFIG)
def save_config_handler(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /wireguard/config — replace config, preserving existing private key.
"""POST /wireguard/config — replace config, preserving existing private keys.
Raises:
ValueError: When request body is missing.
"""
if not body:
raise ValueError("Request body required")
current = _get_config()
current = _get_wireguard_config()
current_key = current.get("interface", {}).get("private_key", "")
if "interface" in body:
body = dict(body)
body["interface"] = dict(body["interface"])
body["interface"].pop("private_key", None)
if current_key:
body.setdefault("interface", {})["private_key"] = current_key
_save_config(body)
refresh_state(["wireguard"])
# Preserve class private keys
if "access_classes" in body:
current_classes = current.get("access_classes", {})
for ck, cv in body.get("access_classes", {}).items():
if isinstance(cv, dict) and ck in current_classes:
cur_class_pk = current_classes[ck].get("private_key", "")
if cur_class_pk and ck in body["access_classes"]:
body["access_classes"][ck]["private_key"] = cur_class_pk
elif "access_classes" not in body:
body["access_classes"] = current.get("access_classes", {})
_save_wireguard_config(body)
emit_and_refresh("wireguard", {"action": "config_saved"})
return {"config_saved": True}
@@ -139,79 +135,266 @@ def patch_config(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""
if not body:
raise ValueError("Request body required")
if "interface" in body:
body = dict(body)
body["interface"] = dict(body["interface"])
body["interface"].pop("private_key", None)
current = _get_config()
# Strip class private keys from patch body
if "access_classes" in body:
for _ck, cv in body["access_classes"].items():
if isinstance(cv, dict):
cv.pop("private_key", None)
current = _get_wireguard_config()
merged = deep_merge(current, body)
_save_config(merged)
refresh_state(["wireguard"])
_save_wireguard_config(merged)
emit_and_refresh("wireguard", {"action": "config_patched"})
return {"config_saved": True}
@registry.register(POST_WIREGUARD_APPLY)
def apply(_request: Any, _body: Any) -> dict[str, Any]:
"""POST /wireguard/apply — render config, write to disk, bring up tunnel via sudo."""
cfg = _get_config()
conf_text = _generate_conf(cfg)
_save_config(cfg)
local_dir = PROJECT_DIR / "data" / "wireguard"
local_dir.mkdir(parents=True, exist_ok=True)
local_tmp = local_dir / "wg0.conf.tmp"
with open(local_tmp, "w") as f:
f.write(conf_text)
os.chmod(local_tmp, 0o600)
run(["cp", "--", str(local_tmp), WG_CONF_PATH], sudo=True)
run(["chown", "root:root", WG_CONF_PATH], sudo=True, check=False)
local_tmp.unlink(missing_ok=True)
run([WG_QUICK_BIN, "up", cfg["interface"]["name"]], sudo=True)
logger.info("WireGuard tunnel '%s' brought up", cfg["interface"]["name"])
refresh_state(["wireguard"])
return {"applied": True}
"""POST /wireguard/apply — render config, write to disk, bring up tunnels via sudo.
In multi-interface mode, applies each class with assigned peers
independently. Falls back to legacy single-interface mode.
"""
cfg = _get_wireguard_config()
classes = cfg.get("access_classes", {})
affected: list[str] = []
for class_key in classes:
class_cfg = classes.get(class_key)
if not class_cfg or not isinstance(class_cfg, dict):
continue
if not _class_peers(cfg, class_key):
continue
try:
conf_text = generate_class_conf(cfg, class_key)
if not conf_text:
continue
except ValueError as e:
logger.warning("Skipping class '%s' during apply: %s", class_key, e)
continue
ifname = _class_interface_name(class_key)
conf_path = _wg_conf_path(ifname)
local_tmp = Path(f"/run/vacuum-wall/{ifname}.conf.tmp")
local_tmp.parent.mkdir(exist_ok=True)
local_tmp.write_text(conf_text)
os.chmod(local_tmp, 0o600)
run(["cp", "--", str(local_tmp), conf_path], sudo=True)
run(["chown", "root:root", conf_path], sudo=True, check=False)
local_tmp.unlink(missing_ok=True)
run([WG_QUICK_BIN, "up", ifname], sudo=True, check=False)
logger.info("WireGuard tunnel '%s' (class '%s') brought up", ifname, class_key)
affected.append(f"{ifname}.conf")
if not affected:
# Legacy single-interface mode
conf_text = generate_conf(cfg)
ifname = cfg["interface"]["name"]
conf_path = _wg_conf_path(ifname)
local_tmp = Path(f"/run/vacuum-wall/{ifname}.conf.tmp")
local_tmp.parent.mkdir(exist_ok=True)
local_tmp.write_text(conf_text)
os.chmod(local_tmp, 0o600)
run(["cp", "--", str(local_tmp), conf_path], sudo=True)
run(["chown", "root:root", conf_path], sudo=True, check=False)
local_tmp.unlink(missing_ok=True)
run([WG_QUICK_BIN, "up", ifname], sudo=True, check=False)
logger.info("WireGuard tunnel '%s' brought up", ifname)
cfg_after = _get_wireguard_config()
stamp_applied(cfg_after)
_save_wireguard_config(cfg_after)
synced = emit_and_refresh("wireguard", {"action": "config_applied"})
return {
"applied": True,
"synced": synced,
"interfaces": affected,
}
@registry.register(POST_WIREGUARD_DOWN)
def down(_request: Any, _body: Any) -> dict[str, Any]:
"""POST /wireguard/down — bring down the WireGuard tunnel via sudo."""
cfg = _get_config()
name = cfg["interface"]["name"]
run([WG_QUICK_BIN, "down", name], sudo=True)
logger.info("WireGuard tunnel '%s' brought down", name)
refresh_state(["wireguard"])
"""POST /wireguard/down — bring down all WireGuard tunnel interfaces."""
cfg = _get_wireguard_config()
classes = cfg.get("access_classes", {})
for class_key in classes:
class_cfg = classes.get(class_key)
if not class_cfg or not isinstance(class_cfg, dict):
continue
ifname = _class_interface_name(class_key)
try:
run([WG_QUICK_BIN, "down", ifname], sudo=True)
logger.info("WireGuard tunnel '%s' brought down", ifname)
except Exception:
pass
# Legacy interface (skip if name matches any class interface)
ifname = cfg["interface"].get("name", "")
if ifname:
class_names = {_class_interface_name(k) for k in classes}
if ifname not in class_names:
try:
run([WG_QUICK_BIN, "down", ifname], sudo=True)
logger.info("WireGuard tunnel '%s' brought down", ifname)
except Exception:
pass
emit_and_refresh("wireguard", {"action": "tunnel_down"})
return {"down": True}
@registry.register(POST_WIREGUARD_CLASSES_UP)
def class_up(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /wireguard/classes/<key>/up — bring up a single class's tunnel.
Raises:
ValueError: When body or key is missing.
NotFoundError: When class does not exist.
"""
if not body:
raise ValueError("Request body required")
class_key = body.get("class_key", "").strip()
if not class_key:
raise ValueError("'class_key' is required")
cfg = _get_wireguard_config()
class_cfg = cfg.get("access_classes", {}).get(class_key)
if not class_cfg:
raise NotFoundError(f"Access class '{class_key}' not found")
conf_text = generate_class_conf(cfg, class_key)
if not conf_text:
raise ValueError(f"No peers assigned to class '{class_key}'")
ifname = _class_interface_name(class_key)
conf_path = _wg_conf_path(ifname)
local_tmp = Path(f"/run/vacuum-wall/{ifname}.conf.tmp")
local_tmp.parent.mkdir(exist_ok=True)
local_tmp.write_text(conf_text)
os.chmod(local_tmp, 0o600)
run(["cp", "--", str(local_tmp), conf_path], sudo=True)
run(["chown", "root:root", conf_path], sudo=True, check=False)
local_tmp.unlink(missing_ok=True)
run([WG_QUICK_BIN, "up", ifname], sudo=True, check=False)
logger.info("WireGuard class '%s' tunnel '%s' brought up", class_key, ifname)
emit_and_refresh("wireguard", {"action": "class_up", "class_key": class_key})
return {"up": True, "interface": ifname}
@registry.register(DELETE_WIREGUARD_CLASSES_DOWN)
def class_down(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""DELETE /wireguard/classes/<key>/down — bring down a single class's tunnel.
Raises:
ValueError: When body or key is missing.
"""
if not body:
raise ValueError("Request body required")
class_key = body.get("class_key", "").strip()
if not class_key:
raise ValueError("'class_key' is required")
cfg = _get_wireguard_config()
if class_key not in cfg.get("access_classes", {}):
raise NotFoundError(f"Access class '{class_key}' not found")
ifname = _class_interface_name(class_key)
try:
run([WG_QUICK_BIN, "down", ifname], sudo=True)
logger.info("WireGuard class '%s' tunnel '%s' brought down", class_key, ifname)
except Exception:
pass
emit_and_refresh("wireguard", {"action": "class_down", "class_key": class_key})
return {"down": True, "interface": ifname}
@registry.register(GET_WIREGUARD_STATUS)
def status(_request: Any, _body: Any) -> dict[str, Any]:
"""GET /wireguard/status — return current WireGuard status from cache."""
wg = _get_wg_state()
if wg:
return wg.get("status", {"up": False, "interface": {}, "peers": []})
return {"up": False, "interface": {}, "peers": []}
return wg.get(
"status", {"up": False, "interface": {}, "peers": [], "classes": {}}
)
return {"up": False, "interface": {}, "peers": [], "classes": {}}
@registry.register(POST_WIREGUARD_INITIALIZE)
def initialize(_request: Any, _body: Any) -> dict[str, Any]:
"""POST /wireguard/initialize — generate keypair and store in config (idempotent)."""
cfg = _get_config()
"""POST /wireguard/initialize — generate keypair and store in config (idempotent).
Also generates key pairs for each access class interface.
"""
cfg = _get_wireguard_config()
if cfg["interface"].get("private_key"):
_ensure_access_classes(cfg)
# Generate keys for classes that need them (idempotent, saves internally)
for class_key in cfg.get("access_classes", {}):
generate_class_keypair(class_key)
return {"initialized": False, "reason": "already initialized"}
res = run_proc([WG_BIN, "genkey"], sudo=True)
private_key = res.stdout.strip()
res2 = run_proc([WG_BIN, "pubkey"], sudo=True, input=private_key)
public_key = res2.stdout.strip()
cfg["interface"]["private_key"] = private_key
cfg["interface"]["public_key"] = public_key
_save_config(cfg)
logger.info("WireGuard initialised (pubkey=%s...)", public_key[:16])
refresh_state(["wireguard"])
# Fresh init
priv, pub = generate_keypair()
cfg["interface"]["private_key"] = priv
cfg["interface"]["public_key"] = pub
_ensure_access_classes(cfg)
# Generate class keys
for class_key in cfg.get("access_classes", {}):
generate_class_keypair(class_key)
_save_wireguard_config(cfg)
logger.info("WireGuard initialised (pubkey=%s...)", pub[:16])
emit_and_refresh("wireguard", {"action": "initialized"})
safe = dict(cfg)
safe["interface"] = dict(safe["interface"])
safe["interface"].pop("private_key", None)
for ck, cv in safe.get("access_classes", {}).items():
if isinstance(cv, dict):
safe["access_classes"][ck] = dict(cv)
safe["access_classes"][ck].pop("private_key", None)
return {"initialized": True, "config": safe}
@registry.register(POST_WIREGUARD_CLASS_INIT_KEYS)
def init_class_keys(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /wireguard/classes/keys/<key> — generate keypair for a class.
Raises:
ValueError: When body or key is missing.
NotFoundError: When class does not exist.
"""
if not body:
raise ValueError("Request body required")
class_key = body.get("class_key", "").strip()
if not class_key:
raise ValueError("'class_key' is required")
cfg = _get_wireguard_config()
if class_key not in cfg.get("access_classes", {}):
raise NotFoundError(f"Access class '{class_key}' not found")
class_cfg = cfg["access_classes"][class_key]
if class_cfg.get("private_key"):
return {
"generated": False,
"class_key": class_key,
"reason": "already has keys",
}
_, pub = generate_class_keypair(class_key)
return {"generated": True, "class_key": class_key, "public_key": pub}
@registry.register(POST_WIREGUARD_PEERS_ADD)
def add_peer(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /wireguard/peers/add — add new peer or update existing one.
@@ -224,33 +407,40 @@ def add_peer(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
name = body.get("name", "").strip()
if not name:
raise ValueError("'name' is required")
cfg = _get_config()
cfg = _get_wireguard_config()
peers = cfg.setdefault("peers", {})
allowed_ips = body.get("allowed_ips", [])
if name in peers:
peer = peers[name]
peer["endpoint"] = body.get("endpoint")
peer["allowed_ips"] = allowed_ips
peer["persistent_keepalive"] = body.get("persistent_keepalive")
if body.get("preshared_key") is not None:
if "endpoint" in body:
peer["endpoint"] = body["endpoint"]
if "allowed_ips" in body:
peer["allowed_ips"] = body["allowed_ips"]
if "persistent_keepalive" in body:
peer["persistent_keepalive"] = body["persistent_keepalive"]
if "preshared_key" in body:
peer["preshared_key"] = body["preshared_key"]
if "description" in body:
peer["description"] = body["description"]
if "access_class" in body:
peer["access_class"] = body["access_class"]
logger.info("WireGuard peer '%s' updated", name)
_peer_action = "peer_updated"
else:
res = run_proc([WG_BIN, "genkey"], sudo=True)
priv = res.stdout.strip()
res2 = run_proc([WG_BIN, "pubkey"], sudo=True, input=priv)
pub = res2.stdout.strip()
priv, pub = generate_keypair()
peers[name] = {
"public_key": pub,
"private_key": priv,
"endpoint": body.get("endpoint"),
"allowed_ips": allowed_ips,
"allowed_ips": body.get("allowed_ips", []),
"persistent_keepalive": body.get("persistent_keepalive"),
"preshared_key": body.get("preshared_key"),
"description": body.get("description", ""),
"access_class": body.get("access_class"),
}
logger.info("WireGuard peer '%s' added", name)
_save_config(cfg)
refresh_state(["wireguard"])
_peer_action = "peer_added"
_save_wireguard_config(cfg)
emit_and_refresh("wireguard", {"action": _peer_action, "peer_name": name})
peer_out = dict(peers[name])
peer_out.pop("private_key", None)
return peer_out
@@ -269,14 +459,14 @@ def remove_peer(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
name = body.get("name", "").strip()
if not name:
raise ValueError("'name' is required")
cfg = _get_config()
cfg = _get_wireguard_config()
peers = cfg.setdefault("peers", {})
if name not in peers:
raise NotFoundError(f"Peer '{name}' not found")
del peers[name]
_save_config(cfg)
_save_wireguard_config(cfg)
logger.info("WireGuard peer '%s' removed", name)
refresh_state(["wireguard"])
emit_and_refresh("wireguard", {"action": "peer_removed", "peer_name": name})
return {"name": name}
@@ -286,14 +476,7 @@ def list_peers(_request: Any, _body: Any) -> list[dict[str, Any]]:
wg = _get_wg_state()
if wg:
return wg.get("peers", [])
cfg = _get_config()
result: list[dict[str, Any]] = []
for name, info in cfg.get("peers", {}).items():
entry = dict(info)
entry["name"] = name
entry.pop("private_key", None)
result.append(entry)
return result
return _get_wireguard_peers()
@registry.register(GET_WIREGUARD_PEER_STATUS)
@@ -305,10 +488,32 @@ def get_peer_status(_request: Any, _body: Any) -> list[dict[str, Any]]:
return []
@registry.register(GET_WIREGUARD_CLASS_STATUS)
def get_class_status(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""GET /wireguard/classes/<key>/status — return status for a specific class interface.
Raises:
NotFoundError: When class does not exist.
"""
wg = _get_wg_state()
if wg:
classes = wg.get("status", {}).get("classes", {})
if body:
key = body.get("class_key", "")
if key and key not in classes:
raise NotFoundError(f"Access class '{key}' not found")
return classes.get(key, {"up": False, "interface": {}, "peers": []})
return classes
return {"up": False, "interface": {}, "peers": []}
@registry.register(POST_WIREGUARD_GENERATE_CLIENT)
def generate_client_conf(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /wireguard/generate-client — render client-side WireGuard config for a peer.
Uses the peer's access class subnet and port to derive the correct
server address and endpoint port.
Raises:
ValueError: When body, name, or server_endpoint is missing.
NotFoundError: When peer does not exist or has no private key.
@@ -321,30 +526,135 @@ def generate_client_conf(_request: Any, body: dict[str, Any] | None) -> dict[str
server_endpoint = body.get("server_endpoint", "")
if not server_endpoint:
raise ValueError("'server_endpoint' is required")
cfg = _get_config()
cfg = _get_wireguard_config()
if name not in cfg.get("peers", {}):
raise NotFoundError(f"Peer '{name}' not found")
peer = cfg["peers"][name]
client_priv = peer.get("private_key", "")
if not client_priv:
if not peer.get("private_key"):
raise NotFoundError(f"Peer '{name}' has no private key")
iface = cfg["interface"]
sorted_peers = sorted(cfg.get("peers", {}).keys())
peer_index = sorted_peers.index(name) + 2
srv_addr = iface["addresses"][0] if iface["addresses"] else "10.137.0.1/24"
addr_part, prefix = srv_addr.rsplit("/", 1)
prefix_base = addr_part.rsplit(".", 1)[0]
client_addr = f"{prefix_base}.{peer_index}/{prefix}"
tmpl = ENV.get_template("wireguard-client.conf")
conf = tmpl.render(
timestamp=datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
conf = _gen_client_conf(
peer_name=name,
client_priv=client_priv,
client_addr=client_addr,
server_pubkey=iface.get("public_key", ""),
server_endpoint=server_endpoint,
allowed_ips=peer.get("allowed_ips", ["0.0.0.0/0"]),
preshared_key=peer.get("preshared_key"),
persistent_keepalive=peer.get("persistent_keepalive"),
server_pubkey=cfg["interface"].get("public_key"),
)
return {"config": conf}
# ---------------------------------------------------------------------------
# Access Classes
@registry.register(GET_WIREGUARD_CLASSES)
def list_classes(_request: Any, _body: Any) -> dict[str, Any]:
"""GET /wireguard/classes — return access classes (with private keys stripped)."""
cfg = _get_wireguard_config()
classes = cfg.get("access_classes", {})
safe: dict[str, Any] = {}
for k, v in classes.items():
if isinstance(v, dict):
entry = dict(v)
entry.pop("private_key", None)
safe[k] = entry
else:
safe[k] = v
return safe
@registry.register(POST_WIREGUARD_CLASSES)
def create_class(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""POST /wireguard/classes — create a new access class with phase-2 fields.
Raises:
ValueError: When body is missing or key conflicts.
"""
if not body:
raise ValueError("Request body required")
key = body.get("key", "").strip()
if not key:
raise ValueError("'key' is required")
if not key.isalnum() or not key.islower():
raise ValueError("'key' must be lowercase alphanumeric")
name = body.get("name", key)
description = body.get("description", "")
cfg = _get_wireguard_config()
classes = cfg.setdefault("access_classes", {})
if key in classes:
raise ConflictError(f"Access class '{key}' already exists")
classes[key] = {
"name": name,
"description": description,
"subnet": body.get("subnet"),
"listen_port": body.get("listen_port"),
"lan_access": body.get("lan_access", False),
"private_key": "",
"public_key": "",
}
_save_wireguard_config(cfg)
emit_and_refresh("wireguard", {"action": "class_created"})
out = dict(classes[key])
out.pop("private_key", None)
return out
@registry.register(PATCH_WIREGUARD_CLASSES)
def update_class(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""PATCH /wireguard/classes/<key> — update an access class.
Raises:
ValueError: When body is missing.
NotFoundError: When class does not exist.
"""
if not body:
raise ValueError("Request body required")
key = body.get("key", "").strip()
if not key:
raise ValueError("'key' is required")
cfg = _get_wireguard_config()
classes = cfg.setdefault("access_classes", {})
if key not in classes:
raise NotFoundError(f"Access class '{key}' not found")
class_cfg = classes[key]
for field in ("name", "description", "subnet", "listen_port", "lan_access"):
if field in body:
class_cfg[field] = body[field]
_save_wireguard_config(cfg)
emit_and_refresh("wireguard", {"action": "class_updated"})
out = dict(classes[key])
out.pop("private_key", None)
return {"key": key, **out}
@registry.register(DELETE_WIREGUARD_CLASSES)
def delete_class(_request: Any, body: dict[str, Any] | None) -> dict[str, Any]:
"""DELETE /wireguard/classes/<key> — remove an access class.
Raises:
ValueError: When body is missing.
NotFoundError: When class does not exist.
ConflictError: When peers still reference the class.
"""
if not body:
raise ValueError("Request body required")
key = body.get("key", "").strip()
if not key:
raise ValueError("'key' is required")
cfg = _get_wireguard_config()
classes = cfg.setdefault("access_classes", {})
if key not in classes:
raise NotFoundError(f"Access class '{key}' not found")
# Check if any peers reference this class
peers_reusing = [
name
for name, info in cfg.get("peers", {}).items()
if info.get("access_class") == key
]
if peers_reusing:
raise ConflictError(
f"Cannot delete class '{key}': {len(peers_reusing)} peer(s) reference it: {', '.join(peers_reusing)}"
)
del classes[key]
_save_wireguard_config(cfg)
emit_and_refresh("wireguard", {"action": "class_deleted"})
return {"key": key}
+66 -1
View File
@@ -29,6 +29,15 @@ PathLike = str | Endpoint
def _ep(method: str, path: str) -> Endpoint:
"""Construct a frozen (method, path) endpoint tuple.
Args:
method: HTTP method string (e.g. ``'GET'``).
path: URL path pattern.
Returns:
Endpoint tuple suitable for ``registry.register()`` and client calls.
"""
return (method, path)
@@ -45,6 +54,10 @@ POST_NGINX_TEST: Endpoint = _ep("POST", "/nginx/test")
POST_NGINX_SSL_APPLY: Endpoint = _ep("POST", "/nginx/ssl-apply")
POST_NGINX_RELOAD: Endpoint = _ep("POST", "/nginx/reload")
GET_NGINX_BACKENDS: Endpoint = _ep("GET", "/nginx/backends")
PATCH_NGINX_BACKENDS: Endpoint = _ep("PATCH", "/nginx/backends")
POST_NGINX_BACKENDS_ADD: Endpoint = _ep("POST", "/nginx/backends/add")
DELETE_NGINX_BACKENDS_REMOVE: Endpoint = _ep("DELETE", "/nginx/backends/remove")
# ---- Firewall ----
GET_FIREWALL_INTERFACES: Endpoint = _ep("GET", "/firewall/interfaces")
@@ -86,6 +99,20 @@ DELETE_WIREGUARD_PEERS_REMOVE: Endpoint = _ep("DELETE", "/wireguard/peers/remove
GET_WIREGUARD_PEERS: Endpoint = _ep("GET", "/wireguard/peers")
GET_WIREGUARD_PEER_STATUS: Endpoint = _ep("GET", "/wireguard/peer-status")
POST_WIREGUARD_GENERATE_CLIENT: Endpoint = _ep("POST", "/wireguard/generate-client")
GET_WIREGUARD_CLASSES: Endpoint = _ep("GET", "/wireguard/classes")
POST_WIREGUARD_CLASSES: Endpoint = _ep("POST", "/wireguard/classes")
PATCH_WIREGUARD_CLASSES: Endpoint = _ep("PATCH", "/wireguard/classes/<key>")
DELETE_WIREGUARD_CLASSES: Endpoint = _ep("DELETE", "/wireguard/classes/<key>")
POST_WIREGUARD_CLASSES_UP: Endpoint = _ep("POST", "/wireguard/classes/<class_key>/up")
DELETE_WIREGUARD_CLASSES_DOWN: Endpoint = _ep(
"DELETE", "/wireguard/classes/<class_key>/down"
)
GET_WIREGUARD_CLASS_STATUS: Endpoint = _ep(
"GET", "/wireguard/classes/<class_key>/status"
)
POST_WIREGUARD_CLASS_INIT_KEYS: Endpoint = _ep(
"POST", "/wireguard/classes/keys/<class_key>"
)
# ---- ACME / Certs ----
GET_ACME_LIST: Endpoint = _ep("GET", "/acme/list")
@@ -94,6 +121,7 @@ POST_ACME_VALIDATE: Endpoint = _ep("POST", "/acme/validate")
POST_ACME_ISSUE: Endpoint = _ep("POST", "/acme/issue")
GET_ACME_ISSUE_STATUS: Endpoint = _ep("GET", "/acme/issue/status")
POST_ACME_RENEW: Endpoint = _ep("POST", "/acme/renew")
GET_ACME_RENEW_STATUS: Endpoint = _ep("GET", "/acme/renew/status")
DELETE_ACME_REMOVE: Endpoint = _ep("DELETE", "/acme/remove")
POST_ACME_EMAIL: Endpoint = _ep("POST", "/acme/email")
GET_ACME_EMAIL: Endpoint = _ep("GET", "/acme/email")
@@ -140,12 +168,49 @@ GET_LOGS_NGINX_ERROR: Endpoint = _ep("GET", "/logs/nginx/error")
GET_LOGS_DNSMASQ: Endpoint = _ep("GET", "/logs/dnsmasq")
GET_LOGS_APP: Endpoint = _ep("GET", "/logs/app")
# ---- Authentication ----
POST_AUTH_LOGIN: Endpoint = _ep("POST", "/auth/login")
POST_AUTH_LOGOUT: Endpoint = _ep("POST", "/auth/logout")
POST_AUTH_REFRESH: Endpoint = _ep("POST", "/auth/refresh")
GET_AUTH_SESSION: Endpoint = _ep("GET", "/auth/session")
POST_AUTH_PASSWORD: Endpoint = _ep("POST", "/auth/password")
# User admin
GET_AUTH_USERS: Endpoint = _ep("GET", "/auth/users")
POST_AUTH_USER_CREATE: Endpoint = _ep("POST", "/auth/users")
POST_AUTH_USER_UPDATE: Endpoint = _ep("POST", "/auth/users/<username>")
DELETE_AUTH_USER: Endpoint = _ep("DELETE", "/auth/users/<username>")
# WebAuthn
POST_AUTH_WEBAUTHN_REGISTER_BEGIN: Endpoint = _ep(
"POST", "/auth/webauthn/register-begin"
)
POST_AUTH_WEBAUTHN_REGISTER_FINISH: Endpoint = _ep(
"POST", "/auth/webauthn/register-finish"
)
POST_AUTH_WEBAUTHN_AUTHENTICATE_BEGIN: Endpoint = _ep(
"POST", "/auth/webauthn/authenticate-begin"
)
POST_AUTH_WEBAUTHN_AUTHENTICATE_FINISH: Endpoint = _ep(
"POST", "/auth/webauthn/authenticate-finish"
)
GET_AUTH_WEBAUTHN_CREDENTIALS: Endpoint = _ep("GET", "/auth/webauthn/credentials")
GET_AUTH_WEBAUTHN_CREDENTIAL_COUNTS: Endpoint = _ep(
"GET", "/auth/webauthn/credential-counts"
)
DELETE_AUTH_WEBAUTHN_CREDENTIAL: Endpoint = _ep(
"DELETE", "/auth/webauthn/creds/<credential_id>"
)
GET_AUTH_WEBAUTHN_CAPABLE: Endpoint = _ep("GET", "/auth/webauthn/capable")
# ---- Server infra (not going through client) ----
GET_HEALTH: Endpoint = _ep("GET", "/health")
GET_STATUS_ALL: Endpoint = _ep("GET", "/status/all")
POST_STATUS_REFRESH: Endpoint = _ep("POST", "/status/refresh")
GET_WS: Endpoint = _ep("GET", "/ws")
POST_BATCH: Endpoint = _ep("POST", "/batch")
GET_STATUS_PENDING: Endpoint = _ep("GET", "/status/pending")
POST_STATUS_APPLY_ALL: Endpoint = _ep("POST", "/status/apply-all")
POST_STATUS_CANCEL_ALL: Endpoint = _ep("POST", "/status/cancel-all")
GET_SYSTEM_METRICS: Endpoint = _ep("GET", "/system/metrics")
# Collect all endpoint module-level constants for __all__ verification
_all_endpoints = [
+270 -59
View File
@@ -9,14 +9,18 @@ import hashlib
import json
import logging
import os
import re
import signal
import time
from collections.abc import Callable
from pathlib import Path
from typing import Any
from aiohttp import web
import daemon.collectors # noqa: F401 (registers state collectors)
from daemon.iface import PathLike
from lib.auth import blacklist_expired
from lib.state import _DEFAULT_POLL_INTERVALS
from lib.state import state as state_store
@@ -34,7 +38,15 @@ if _RAW_POLL:
if ":" in pair:
name, _, val = pair.partition(":")
try:
_POLL_OVERRIDE[name.strip()] = int(val.strip())
parsed = int(val.strip())
if parsed <= 0:
logger.warning(
"Invalid poll interval value %r for %r (must be > 0), skipping",
val,
name,
)
continue
_POLL_OVERRIDE[name.strip()] = parsed
except ValueError:
logger.warning(
"Invalid poll interval value %r for %r, skipping", val, name
@@ -94,6 +106,7 @@ class Registry:
method = ep_method
def decorator(fn: Callable) -> Callable:
"""Inner decorator that stores *fn* in the registry and attaches Handler metadata."""
self._routes[(method.upper(), path)] = fn # type: ignore[arg-type]
fn._handler = Handler(method, path) # type: ignore[attr-defined,reportArgumentType]
return fn
@@ -133,22 +146,33 @@ class Registry:
registry = Registry()
def refresh_state(subsystems: list[str] | None = None) -> None:
def refresh_state(subsystems: list[str] | None = None, bump: bool = True) -> None:
"""Refresh the pre-computed state for the given subsystems (or all).
Args:
subsystems: List of subsystem names to refresh. If None, all subsystems are refreshed.
bump: Bump the version counter for each refreshed subsystem.
``refresh_status`` passes ``False`` — versions advance on
structural poll diffs and on mutation-triggered refreshes only.
"""
state_store.populate(subsystems)
targets = subsystems or state_store.SUBSYSTEMS
for name in targets:
state_store.bump(name)
if bump:
for name in targets:
state_store.bump(name)
try:
asyncio.get_running_loop()
except RuntimeError:
pass
else:
task = asyncio.create_task(broadcast_versions())
# Broadcast each refreshed subsystem individually (gather for parallelism)
async def _broadcast_all():
await asyncio.gather(
*[broadcast_versions(name) for name in targets],
return_exceptions=True,
)
task = asyncio.create_task(_broadcast_all())
task.add_done_callback(_ws_tasks.discard)
_ws_tasks.add(task)
@@ -228,7 +252,8 @@ async def _handle_request(request: web.Request) -> web.Response:
# Build body — merge order (highest wins): path params > JSON body > query params.
# Path params come from the URL path (e.g. /interfaces/eth0) and should not
# be overridable by body or query parameters.
# be overridable by body or query parameters. This prevents callers from
# spoofing path-scoped parameters via request body.
body: dict[str, Any] | None = pat_params if pat_params else None
if request.content_type == "application/json":
try:
@@ -340,7 +365,6 @@ def create_app() -> web.Application:
"""
app = web.Application()
app.router.add_route("GET", "/health", _health)
app.router.add_route("GET", "/status/all", get_status_all)
app.router.add_route("POST", "/status/refresh", refresh_status)
app.router.add_route("POST", "/batch", _handle_batch)
app.router.add_route("GET", "/ws", _handle_ws)
@@ -348,23 +372,102 @@ def create_app() -> web.Application:
return app
# JWTs are dot-joined base64url segments — a subset of the RFC 6455 token
# character set. Browsers therefore carry the access token as the
# Sec-WebSocket-Protocol subprotocol name itself (see websocket.js); the
# "Bearer <token>" form is not a valid subprotocol (space is not a token
# character) and is rejected by the WebSocket constructor.
_JWT_SUBPROTOCOL_RE = re.compile(r"^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$")
def _extract_ws_token(subprotocols: list[str]) -> tuple[str | None, str | None]:
"""Extract the auth JWT from parsed Sec-WebSocket-Protocol names.
Accepts the raw JWT as a subprotocol name (bundled client) or the
legacy "Bearer <token>" form (non-browser clients that can send spaces).
Args:
subprotocols: Parsed subprotocol names (already stripped/split).
Returns:
``(token, matched_subprotocol)`` or ``(None, None)`` if no usable
auth subprotocol was found.
"""
for proto in subprotocols:
if _JWT_SUBPROTOCOL_RE.fullmatch(proto):
return proto, proto
for proto in subprotocols:
if proto.startswith("Bearer "):
token = proto[7:].strip()
if token:
return token, proto
return None, None
# WebSocket subscribers
_ws_subscribers: set[web.WebSocketResponse] = set()
_ws_tasks: set[asyncio.Task[None]] = set()
_poll_tasks: set[asyncio.Task[None]] = set()
_last_blacklist_cleanup: float = 0
_last_blacklist_cleanup_lock: asyncio.Lock | None = None
async def _handle_ws(request: web.Request) -> web.Response:
"""WebSocket endpoint for real-time state change notifications.
"""WebSocket endpoint for real-time state streaming.
On connect: sends current versions. On state change: broadcasts
updated subsystem versions. Clients disconnect to unsubscribe.
On connect: sends a full state snapshot of every subsystem
({type: snapshot, data: {subsystem: state, …}}). On state change:
broadcasts a data-carrying per-subsystem delta (versions or tick).
Clients disconnect to unsubscribe.
Authentication: JWT access token passed via:
1. WebSocket subprotocol header — the bundled client sends the raw JWT
as the subprotocol name (Sec-WebSocket-Protocol must carry a valid
RFC 6455 token; a JWT is one, "Bearer <token>" is not).
2. "Bearer <token>" subprotocol — legacy form for non-browser clients.
3. X-Auth-Token header — fallback for custom nginx setups that inject
it (not set by the bundled nginx config).
"""
ws = web.WebSocketResponse()
from aiohttp import hdrs
from lib.auth import validate_token
token_param = None
matched_proto = None
# Prefer the subprotocol header. Sec-WebSocket-Protocol is a
# comma-separated list; parse it the same way aiohttp's own handshake
# does (Request has no subprotocol helper).
protocol_header = request.headers.get(hdrs.SEC_WEBSOCKET_PROTOCOL, "")
subprotocols = [p.strip() for p in protocol_header.split(",") if p.strip()]
token_param, matched_proto = _extract_ws_token(subprotocols)
if token_param is None:
token_param = request.headers.get("X-Auth-Token")
if token_param is None:
return web.json_response(
{"ok": False, "error": "authentication required"}, status=401
)
# Validate token. Session binding is skipped because browsers cannot send
# custom headers on WebSocket connections (no X-Session-Id available).
payload = validate_token(
token_param,
token_type="access",
)
if payload is None:
return web.json_response({"ok": False, "error": "unauthorized"}, status=401)
# Negotiate only the matched auth subprotocol (or all if token came from header)
ws = web.WebSocketResponse(
protocols=[matched_proto] if matched_proto else subprotocols
)
await ws.prepare(request)
_ws_subscribers.add(ws)
await ws.send_json({"type": "init", "versions": state_store.get_versions()})
snapshot = state_store.get_snapshot()
await ws.send_json({"type": "snapshot", "data": snapshot})
try:
async for msg in ws:
@@ -378,39 +481,69 @@ async def _handle_ws(request: web.Request) -> web.Response:
return ws
async def broadcast_versions() -> None:
"""Broadcast updated subsystem versions to all WebSocket clients."""
updated = state_store.get_updated_versions()
if not updated or not _ws_subscribers:
async def _send_all(message: str) -> None:
"""Send a message string to all WS subscribers, removing dead ones."""
dead: set[web.WebSocketResponse] = set()
for ws in _ws_subscribers:
try:
await ws.send_str(message)
except Exception:
dead.add(ws)
_ws_subscribers.difference_update(dead)
if dead:
logger.warning("Removed %d dead WS subscribers", len(dead))
async def broadcast_versions(subsystem: str) -> None:
"""Send {type: versions} + full subsystem data to WS clients.
NOTE: Does NOT call state_store.bump(). Callers who need version bumps
(e.g. refresh_state) call bump themselves. poll_loop bumps before calling.
No legacy `updated` field is emitted (no backward compat) — version
counters still advance but are not sent over the wire.
"""
data = state_store.get(subsystem)
if data is None:
# Collector failed during the triggering populate/refresh — populate()
# cleared this subsystem to None (state.py). Skip the broadcast:
# a null payload would overwrite good client data. The next successful
# poll or mutation broadcasts the real value.
return
data = json.dumps({"type": "versions", "updated": updated})
dead: set[web.WebSocketResponse] = set()
for ws in _ws_subscribers:
try:
await ws.send_str(data)
except Exception:
dead.add(ws)
_ws_subscribers.difference_update(dead)
if dead:
logger.warning("Removed %d dead WS subscribers", len(dead))
message = json.dumps(
{
"type": "versions",
"subsystem": subsystem,
"data": data,
}
)
await _send_all(message)
async def broadcast_tick(subsystems: list[str]) -> None:
"""Broadcast a lightweight tick to WS clients without version payload."""
data = json.dumps({"type": "tick", "subsystems": subsystems})
dead: set[web.WebSocketResponse] = set()
for ws in _ws_subscribers:
try:
await ws.send_str(data)
except Exception:
dead.add(ws)
_ws_subscribers.difference_update(dead)
if dead:
logger.warning("Removed %d dead WS subscribers", len(dead))
async def broadcast_tick(subsystem: str) -> None:
"""Send {type: tick} with the changed subsystem data.
No bump — tick is volatile-only; the version counter only bumps on
structural changes.
"""
data = state_store.get(subsystem)
message = json.dumps(
{
"type": "tick",
"subsystem": subsystem,
"data": data,
}
)
await _send_all(message)
async def _poll_loop(subsystem: str, interval: int) -> None:
"""Periodically poll a subsystem for state changes and broadcast as needed."""
global _last_blacklist_cleanup, _last_blacklist_cleanup_lock
# Lazy-init lock (requires running event loop)
if _last_blacklist_cleanup_lock is None:
_last_blacklist_cleanup_lock = asyncio.Lock()
offset = int(hashlib.md5(subsystem.encode()).hexdigest(), 16) % interval
await asyncio.sleep(offset)
while True:
@@ -418,9 +551,15 @@ async def _poll_loop(subsystem: str, interval: int) -> None:
structural, volatile = state_store.poll(subsystem)
if structural:
state_store.bump(subsystem)
await broadcast_versions()
await broadcast_versions(subsystem) # now per-subsystem, carries data
elif volatile:
await broadcast_tick([subsystem])
await broadcast_tick(subsystem) # now per-subsystem, carries data
# Periodic blacklist cleanup — coordinated across all poll loops
async with _last_blacklist_cleanup_lock:
now = time.time()
if now - _last_blacklist_cleanup >= 60:
blacklist_expired()
_last_blacklist_cleanup = now
except asyncio.CancelledError:
raise
except Exception:
@@ -452,15 +591,6 @@ async def _health(_request: web.Request) -> web.Response:
return ok({"pid": os.getpid(), "socket": str(SOCKET_PATH)})
async def get_status_all(_request: web.Request) -> web.Response:
"""Return the entire state snapshot in one call.
Returns:
JSON response containing state data for all subsystems.
"""
return ok({name: state_store.get(name) for name in state_store.SUBSYSTEMS})
async def refresh_status(_request: web.Request) -> web.Response:
"""Re-collect all state from system.
@@ -474,11 +604,13 @@ async def refresh_status(_request: web.Request) -> web.Response:
body = await _request.json()
except (json.JSONDecodeError, ValueError):
body = None
subsystems = None
if body and "subsystems" in body:
subsystems = body["subsystems"]
state_store.populate(subsystems)
return ok({name: state_store.get(name) for name in state_store.SUBSYSTEMS})
subsystems = body.get("subsystems") if body else None
# Deliberately no version bump — versions advance on structural poll
# diffs and on refresh_state() only.
refresh_state(subsystems, bump=False)
targets = subsystems or state_store.SUBSYSTEMS
snapshot = {name: state_store.get(name) for name in targets}
return ok(snapshot)
async def _catch_all(request: web.Request) -> web.Response:
@@ -501,11 +633,14 @@ def _register_routes() -> None:
"""
from daemon.handlers import (
acme, # noqa: F401
auth, # noqa: F401
dnsmasq, # noqa: F401
firewall, # noqa: F401
logs, # noqa: F401
network, # noqa: F401
nginx, # noqa: F401
status, # noqa: F401
system, # noqa: F401
wireguard, # noqa: F401
)
@@ -520,6 +655,14 @@ def main() -> None:
setup_logging()
# Startup checks
try:
from lib import webauthn as lib_webauthn
lib_webauthn.check_webauthn_config()
except Exception:
pass # ignore if webauthn module import failed
_register_routes()
app = create_app()
@@ -532,13 +675,55 @@ def main() -> None:
loop = asyncio.new_event_loop()
def _on_shutdown(_sig: int) -> None:
def _teardown_exception_handler(_loop, context) -> None:
# Swallow teardown noise ("Task was destroyed but it is pending",
# in-flight task exceptions on SIGTERM) instead of the default
# logging-error tracebacks.
logger.debug("Suppressed teardown exception: %s", context)
async def _shutdown() -> None:
"""Graceful shutdown: stop accepting, drain in-flight work, teardown.
Bounded grace periods + a suppressed exception handler during the
teardown window avoid the "Task was destroyed but it is pending" and
logging-error tracebacks that otherwise appear on SIGTERM.
"""
logger.info("Shutting down daemon...")
_stop_polling()
loop.stop()
# Suppress the default exception handler during teardown so that
# cancelling in-flight tasks does not spew tracebacks on SIGTERM.
prev_handler = loop.get_exception_handler()
loop.set_exception_handler(_teardown_exception_handler)
try:
# Stop accepting new connections (also waits for open sockets,
# bounded so a stuck WebSocket can't hang shutdown).
try:
await asyncio.wait_for(runner.cleanup(), timeout=5)
except TimeoutError:
logger.warning("Runner cleanup timed out, abandoning")
# Give in-flight request/WS tasks a bounded grace period to
# finish; cancel anything still pending so they are not
# "destroyed but pending" when the loop closes.
pending = [
t
for t in asyncio.all_tasks()
if t is not asyncio.current_task() and not t.done()
]
if pending:
_, still_pending = await asyncio.wait(pending, timeout=3)
for t in still_pending:
t.cancel()
if still_pending:
await asyncio.wait(still_pending, timeout=1)
finally:
loop.set_exception_handler(prev_handler)
if Path(socket_path).exists():
os.unlink(socket_path)
logger.info("vacuum-walld stopped")
loop.call_soon(loop.stop)
for sig in (signal.SIGTERM, signal.SIGINT):
loop.add_signal_handler(sig, _on_shutdown, sig)
loop.add_signal_handler(sig, lambda: loop.create_task(_shutdown()))
runner = web.AppRunner(app)
loop.run_until_complete(runner.setup())
@@ -549,6 +734,31 @@ def main() -> None:
os.chmod(socket_path, 0o660)
# Import system configs → JSON (blocking — OK at startup)
from lib.system_import import import_all
reconciled = import_all()
if reconciled:
logger.info("Reconciled subsystems: %s", ", ".join(reconciled))
# Filesystem bootstrap after the import (which must see absent config
# files to adopt live system state on first start): create runtime
# directories and persist the one-shot nginx legacy-format migration.
from lib.bootstrap import bootstrap
bootstrap()
# Reopen group access on the ACME home before the first acme.sh
# collection: a tree left owner-only by a prior run (e.g. a manual
# run as the WebUI user) would otherwise fail every daemon acme.sh
# call until the next issue/renew. Never fatal at startup.
try:
from daemon.handlers.acme import normalize_acme_home
normalize_acme_home()
except Exception:
logger.warning("ACME home normalization failed at startup", exc_info=True)
# Populate state from system (blocking — OK at startup)
logger.info("Populating system state...")
state_store.populate()
@@ -562,10 +772,11 @@ def main() -> None:
try:
loop.run_forever()
finally:
loop.run_until_complete(runner.cleanup())
# _shutdown() handles cleanup when invoked via signal handler;
# this block is only reached if shutdown didn't happen cleanly
# (e.g., unexpected exit), in which case we unlink the socket.
if Path(socket_path).exists():
os.unlink(socket_path)
logger.info("vacuum-walld stopped")
if __name__ == "__main__":
+1046 -110
View File
File diff suppressed because it is too large Load Diff
+295 -41
View File
@@ -10,19 +10,20 @@ The following describes the path a request takes from an external client to a ba
2. The request arrives at the Vacuum Wall host's WAN interface, assigned to the `external` firewalld zone. A firewall rule allows inbound traffic on port 443 (HTTPS).
3. nginx, listening on port 443, terminates the TLS connection using the domain's certificate.
4. nginx evaluates the `server_name` against the configured server blocks. The matching block is generated from the domain entry in `config/nginx/config.json`.
5. The request is forwarded to the backend service (e.g., `192.168.2.50:8080`) via an `proxy_pass` directive.
6. The backend service processes the request and returns an HTTP response.
7. nginx adds security headers (`X-Content-Type-Options`, `X-Frame-Options`, HSTS, etc.) to the response.
8. nginx encrypts the response with TLS and sends it back to the client through the WAN interface.
5. If the domain has an `auth` block, nginx applies HTTP Basic authentication before proxying. Auth is resolved in the order domain → backend (the effective `auth` is the domain's own, or the referenced backend's if the domain has none), and per-path behavior follows the resolved auth config. Credentials are checked against the generated `data/nginx/.htpasswd` file; unauthenticated requests receive a 401 with a `WWW-Authenticate` challenge. Domains without an `auth` block skip this step entirely.
6. The request is forwarded to the backend service (e.g., `192.168.2.50:8080`) via a `proxy_pass` directive (per-path upstreams resolved from the backend's `paths` config).
7. The backend service processes the request and returns an HTTP response.
8. nginx adds security headers (`X-Content-Type-Options`, `X-Frame-Options`, HSTS, etc.) to the response.
9. nginx encrypts the response with TLS and sends it back to the client through the WAN interface.
For HTTP requests (port 80), nginx returns a 301 redirect to the HTTPS equivalent before any proxying occurs.
### Management WebUI Access (e.g., `<hostname>.local`)
1. A client sends an HTTPS request to the management domain.
2. nginx terminates TLS and checks for HTTP Basic Authentication credentials against the `.htpasswd` file.
3. If authentication succeeds, the request is proxied to `127.0.0.1:9090` where the Flask WebUI is listening.
4. The Flask application processes the request and communicates with the `vacuum-walld` daemon via a Unix socket (`data/daemon.sock`) for any privileged operations.
2. nginx terminates TLS and proxies the request to `127.0.0.1:9090` where the Flask WebUI is listening. No nginx-level authentication is applied.
3. Flask validates the JWT from the `Authorization: Bearer <token>` header together with the `X-Session-Id` header (both are required; the session ID must match the token's `session_id` claim), checks the token against the SQLite blacklist (`data/auth.db`), and verifies per-subsystem permissions before processing the request. Public endpoints (login, token refresh, WebAuthn authenticate) are exempt from validation.
4. The Flask application communicates with the `vacuum-walld` daemon via a Unix socket (`data/daemon.sock`) for any privileged operations.
5. The daemon executes the privileged commands via the sudo whitelist and returns structured results.
6. Flask renders an HTML or JSON response, which nginx returns to the client over the encrypted connection.
@@ -33,34 +34,103 @@ Because Flask binds only to `127.0.0.1`, it is unreachable directly from any ext
The following diagram summarizes how the Flask WebUI communicates with each managed subsystem:
```
External Client ──→ nginx (SSL termination) ──→ Flask WebUI (127.0.0.1:9090)
External Client ──→ nginx (SSL termination; auth_basic only on proxy domains with an `auth` block) ──→ Flask WebUI (127.0.0.1:9090, JWT + X-Session-Id + permission check)
Flask WebUI ──→ daemon/client.py (path resolution, Unix socket) ──→ vacuum-walld (aiohttp server)
Flask WebUI ──→ lib/db.py (abstract DB interface) ──→ SQLite (data/auth.db)
vacuum-walld ──→ daemon/handlers/auth.py ──→ lib/auth.py ──→ JWT operations
vacuum-walld ──→ daemon/handlers/firewall.py ──→ sudo firewall-cmd ──→ firewalld / D-Bus ──→ nftables
vacuum-walld ──→ daemon/handlers/nginx.py ──→ write local .conf files ──→ sudo cp to /etc/nginx/ ──→ sudo nginx -t && sudo nginx -s reload
vacuum-walld ──→ daemon/handlers/dnsmasq.py ──→ render config ──→ sudo tee /etc/dnsmasq.d/vacuum-wall.conf ──→ sudo systemctl reload dnsmasq
vacuum-walld ──→ daemon/handlers/acme.py ──→ acme.sh (subprocess) ──→ deploy hook (daemon API) ──→ ZeroSSL ACME
vacuum-walld ──→ daemon/handlers/wireguard.py ──→ render data/wireguard/wg0.conf ──→ sudo cp to /etc/wireguard/ ──→ sudo wg-quick up wg0
vacuum-walld ──→ daemon/handlers/network.py ──→ render 50-<name>.network ──→ sudo cp to /etc/systemd/network/ ──→ sudo networkctl reload
vacuum-walld ──→ daemon/handlers/nginx.py ──→ render temps in /run/vacuum-wall ──→ sudo cp to /etc/nginx/ ──→ sudo nginx -t && sudo nginx -s reload (SIGHUP)
vacuum-walld ──→ daemon/handlers/dnsmasq.py ──→ render config ──→ /run/vacuum-wall/dnsmasq.tmp ──→ sudo cp to /etc/dnsmasq.d/vacuum-wall.conf ──→ sudo systemctl restart dnsmasq
vacuum-walld ──→ daemon/handlers/acme.py ──→ acme.sh (subprocess) ──→ on issue/renew success: deploy hook (sudo nginx -t && sudo nginx -s reload) ──→ nginx
vacuum-walld ──→ daemon/handlers/wireguard.py ──→ render per-class config (wg-<class>) ──→ /run/vacuum-wall/<ifname>.conf.tmp (0600) ──→ sudo cp to /etc/wireguard/<ifname>.conf ──→ sudo wg-quick up <ifname>
vacuum-walld ──→ daemon/handlers/network.py ──→ render 99-<name>.network ──→ sudo cp to /etc/systemd/network/ ──→ sudo networkctl reload + sudo networkctl reconfigure <iface>
vacuum-walld ──→ daemon/handlers/logs.py ──→ sudo journalctl ──→ systemd journal
vacuum-walld ──→ daemon/handlers/status.py ──→ cross-subsystem apply-all (networkd→firewall→wireguard→dnsmasq→nginx) + cancel-all (revert to last applied)
vacuum-walld ──→ daemon/handlers/system.py ──→ pre-collected /proc metrics (state store)
```
Notes on the diagram:
- The ACME flow terminates at nginx: acme.sh stores certs on disk and the `deploy` step fires the deploy hook (`system/acme-deploy.sh`, installed to `$ACME_HOME/deploy/` by the install script) only after a successful issue or renewal; the hook runs `sudo nginx -t && sudo nginx -s reload`. A Python hook variant (`system/acme-deploy.py`) that instead calls the daemon's `POST /nginx/reload` endpoint exists in the tree but is not the one the install script installs.
- WireGuard multi-interface mode: the config defines `access_classes`; each class with peers gets its own interface `wg-<class>`, rendered to `/etc/wireguard/wg-<class>.conf`. Legacy single-interface mode renders `/etc/wireguard/wg0.conf`.
### Two-User Model with Shared Group
Vacuum Wall uses two distinct system users bridged by a shared group:
- **`vacuum-walld`** (daemon user): Runs the privileged background daemon. Holds the NOPASSWD sudo whitelist for all system-level commands. Runs with `NoNewPrivileges=yes` (satisfiable since sudo is called directly by the daemon process).
- **`vacuum-walld`** (daemon user): Runs the privileged background daemon. Holds the NOPASSWD sudo whitelist for all system-level commands. Runs with `NoNewPrivileges=yes` (satisfiable since sudo is called directly by the daemon process). Its systemd unit declares `RuntimeDirectory=vacuum-wall nginx` (pre-creates `/run/vacuum-wall` and `/run/nginx` before namespace setup) and `LogsDirectory=vacuum-wall` (`/var/log/vacuum-wall`; the management unit declares the same `LogsDirectory`).
- **WebUI user** (default: repo owner in `--dev` mode): Runs the Flask web serving process. Has **zero** sudo access. Communicates with the daemon via a Unix socket at `data/daemon.sock`. Runs with `NoNewPrivileges=yes`.
- **Shared group**: Both users share the WebUI user's primary group. The daemon socket is owned by `vacuum-walld:<group>` with mode `0660`, allowing the web UI user to connect via group permission. The project directory is owned by the WebUI user with group-read+execute, giving the daemon read access to configs and shared files.
- **Shared group**: Both users share the WebUI user's primary group. The daemon socket is owned by `vacuum-walld:<group>` with mode `0660`, allowing the web UI user to connect via group permission. In **production**, the project directory is owned by the **daemon user** with group read+write (`g+rwX`) and the setgid bit on all subdirectories, so the WebUI user can read configs and shared files via the shared group. In **`--dev` mode only**, the project directory stays owned by the repo owner (the WebUI user).
This design isolates privilege escalation entirely within the daemon, so a compromised Flask process cannot invoke sudo directly. The `lib/` modules no longer contain sudo calls; all privileged command execution lives in `daemon/handlers/*.py`.
This design isolates privilege escalation entirely within the daemon, so a compromised Flask process cannot invoke sudo directly. All *mutating* privileged operations live in `daemon/handlers/*.py`, but a few `lib/` code paths still execute sudo and are only ever called from within the daemon process:
**Dev mode variant**: When `install.sh --dev` is used, the repo owner (e.g., `wall`) becomes the WebUI user. The project directory remains owned by the repo owner, preserving git operations and code editing. The daemon user (`vacuum-walld`) has the repo owner's primary group as its own primary group, granting read access to project files. All subdirectories carry the setgid bit (`g+s`) so new files inherit the group regardless of the creator's primary group.
- `lib/common.get_interface_ip``sudo ip -o addr show <iface>` (used by sync subscribers and handlers to backfill gateway addresses)
- `lib/system_import.import_firewall``sudo firewall-cmd --list-all-zones` (startup import only)
- `lib/nginx.test_config``sudo nginx -t` (imported live by `daemon/handlers/acme.py` for ACME pre-flight checks)
- Legacy sudo code in `lib/nginx.py` (install/reload helpers) and `lib/wireguard.py` (legacy apply/down paths)
The `lib/` modules auto-discover the project root at runtime via `Path(__file__).resolve().parent.parent`. This works because `install.sh` performs an editable pip install (`pip install -e .`), keeping module files in the project directory rather than copying them to `site-packages/`.
**Dev mode variant**: When `scripts/install.sh --dev` is used, the repo owner (e.g., `wall`) becomes the WebUI user. The project directory remains owned by the repo owner, preserving git operations and code editing. The daemon user (`vacuum-walld`) has the repo owner's primary group as its own primary group, granting read access to project files. All subdirectories carry the setgid bit (`g+s`) so new files inherit the group regardless of the creator's primary group.
The `lib/` modules auto-discover the project root at runtime via `Path(__file__).resolve().parent.parent`. This works because `scripts/install.sh` performs an editable pip install (`pip install -e .`), keeping module files in the project directory rather than copying them to `site-packages/`.
## JWT Token Model
Vacuum Wall uses JWT-based authentication with access/refresh token rotation. Tokens are stored in browser `sessionStorage` and injected as `Authorization: Bearer <token>` headers. The API never reads cookies — authentication is header-only.
| Token | Lifetime | Storage | Purpose |
|---|---|---|---|
| Access | 5 min on fresh install (config-driven; code fallback 900 s) | sessionStorage / memory | API auth, permission checks |
| Refresh | 7 days | sessionStorage | Token rotation, new access tokens |
Every JWT payload contains `sub` (username), `exp` (expiry), `iat` (issued at), `jti` (unique identifier), and `type` (`"access"` or `"refresh"`). **Access tokens additionally carry `permissions` (per-subsystem permissions) and `session_id` (session binding — always present; a fresh ID is generated if the caller does not supply one). Refresh tokens carry only `session_id`, and only when a session was bound at login — they never carry `permissions`.** The Flask middleware relies on both: it reads `permissions` from the access token and cross-checks the `X-Session-Id` request header against the token's `session_id` claim.
Each user has a unique signing secret stored in the `users.jwt_secret` database column (generated as a 32-byte base64url token via `secrets.token_urlsafe(32)`). This per-user secret model means tokens signed for one user cannot be validated as another user's tokens. Both Flask and daemon processes validate tokens by extracting `sub` from the unverified payload, looking up the user's secret, and verifying the signature with that secret. Expired and blacklisted tokens are rejected against the SQLite `token_blacklist` table (via `data/auth.db`).
Token auto-refresh occurs before expiry. On logout or password change, tokens are blacklisted in the SQLite `token_blacklist` table to prevent reuse. Blacklist cleanup is **probabilistic, not per-refresh**: `blacklist_token()` purges expired entries with a 2% chance on each call, and the daemon's poll loop additionally runs `blacklist_expired()` at most every 60 seconds (coordinated across all poll loops via a shared lock).
## Permission Model
Each user has per-subsystem permissions with two levels:
- **`"read"`** — `GET /api/<subsystem>/*` allowed; `POST`/`PATCH`/`DELETE` rejected with 403
- **`"rw"`** — all HTTP methods allowed for the subsystem
Flask `before_request` middleware enforces permissions by extracting the subsystem name from the blueprint route prefix (e.g., `/api/firewall/``"firewall"`). The middleware checks `request.user.permissions[subsystem]`. If the permission level doesn't match the required level, a 403 response is returned.
The `auth` subsystem controls user management. User CRUD endpoints (`/api/auth/users/*`) require `auth: "rw"` ("admin required").
Login-related endpoints are public (no JWT required): `POST /api/auth/login`, `POST /api/auth/refresh`, `POST /api/auth/webauthn/authenticate-begin`, `POST /api/auth/webauthn/authenticate-finish`. (The SPA root `GET /` is also exempt.) Note that for all *other* API routes the middleware requires **both** the `Authorization: Bearer <token>` and `X-Session-Id` headers — a request with only one of the two is rejected with 401.
## Database Layer
Vacuum Wall uses SQLite for authentication and user management data. Subsystem configuration remains as JSON in `config/*/`.
**Architecture:**
- `lib/db.py` — Query ID constants + abstract `Database` baseclass (no SQL strings)
- `lib/db_sqlite.py``QUERY_MAP` (query_id → SQLite SQL) + concrete implementation
- Subsystems call by **query ID only** — never write SQL
The abstract `Database` baseclass provides:
- Connection caching via `self.conn` property (lazy initialization)
- Prepared statement auto-cache (cached on first use, reused subsequently)
- `query(query_id, params)` — returns row dicts
- `run(query_id, params)` — returns rowcount
- `run_one(query_id, params)` — returns last_insert_id
- `in_transaction()` context manager — provides `BEGIN`/`COMMIT`/`ROLLBACK` with auto-commit suppressed inside
Environment variables (not config files) control database access:
| Env Var | Default | Description |
|---|---|---|
| `VACUUM_WALL_DB_BACKEND` | `sqlite` | Database backend selection |
| `VACUUM_WALL_DB_PATH` | `data/auth.db` | SQLite database file path |
Flask (`webui/server.py`) calls `get_db()` once at startup to open (and initialize) the database. The daemon does **not** call `get_db()` at startup — it reaches the database lazily through `lib.auth` / `lib.auth_users` the first time an auth operation actually runs. Each process opens its own connection to the same DB file. SQLite WAL mode enables concurrent reads; writes are serialized by SQLite.
## Install-Time Templating
System configuration files in `system/` are Jinja2 templates rendered by `install.sh` at install time:
System configuration files in `system/` are Jinja2 templates rendered by `scripts/install.sh` at install time:
- **`systemd/vacuum-wall.service`**, **`systemd/vacuum-walld.service`**, **`systemd/vacuum-wall-acme.service`** — `{{ USER_NAME }}`, `{{ USER_DAEMON_NAME }}`, `{{ USER_GROUP }}`, `{{ PROJECT_DIR }}`, `{{ ACME_HOME }}` are substituted to produce the final systemd unit files installed to `/etc/systemd/system/`. The `PROJECT_DIR` template variable is set from the `INSTALL_DIR` environment variable (defaults to the repo root).
- **`sudoers.d/vacuum-walld`** — `{{ USER_DAEMON_NAME }}` is substituted to produce the sudoers whitelist for the daemon user.
@@ -74,12 +144,12 @@ Vacuum Wall uses a declarative configuration model. Persistent user-facing confi
| Subsystem | Declarative Config | Runtime Data | Rendered Target | State Persistence |
|---|---|---|---|---|
| firewalld | `config/firewall/config.json` | `data/firewall/rules.json` | N/A (commands issued directly to firewalld via D-Bus) | firewalld manages its own persistent state in `/etc/firewalld/`. `config.json` is the declarative source of truth. `rules.json` serves as an automated backup snapshot. |
| firewalld | `config/firewall/config.json` | `data/firewall/rules.json` | N/A (commands issued directly to firewalld via D-Bus) | firewalld manages its own persistent state in `/etc/firewalld/`. `config.json` is the declarative source of truth. `rules.json` is a pre-apply recovery snapshot (`{timestamp, default_zone, zones, config}`) written before every apply; `zones` is the permanent firewalld zone view. |
| dnsmasq | `config/dnsmasq/config.json` | `data/dnsmasq/fragments/` | `/etc/dnsmasq.d/vacuum-wall.conf` | The JSON file is the source of truth. The rendered `.conf` file is overwritten on each apply. |
| nginx | `config/nginx/config.json` | `data/nginx/.htpasswd`, `data/nginx/sites-enabled/` | `data/nginx/sites-enabled/<domain>.conf` + `/etc/nginx/conf.d/vacuum-wall.conf` | All proxy and management domain definitions are derived from the JSON config. Generated `.conf` files are overwritten on each apply. |
| WireGuard | `config/wireguard/config.json` | `data/wireguard/` | `/etc/wireguard/wg0.conf` | The JSON file defines the interface and all peers. The rendered WireGuard config is overwritten on each apply. |
| networkd | `config/network/config.json` | `data/networkd/` | `/etc/systemd/network/50-<name>.network` | The JSON file defines per-interface static addresses, routes, DNS, DHCP, and link settings. Each entry renders to a `50-<name>.network` INI file. Stale files are cleaned on apply. Public DNS servers are auto-synced to dnsmasq upstreams. |
| ACME | N/A (`~/.acme.sh/` managed by acme.sh) | `data/acme/` | Certificate and key files | acme.sh manages its own state, renewal scheduling, and account keys. Vacuum Wall triggers issuance and renewal but does not maintain independent ACME state. |
| WireGuard | `config/wireguard/config.json` | `data/wireguard/` | `/etc/wireguard/<ifname>.conf` — per-class `wg-<class>.conf` in multi-interface mode; legacy single-interface `wg0.conf` | The JSON file defines the interface, `access_classes`, and all peers. In multi-interface mode each class with assigned peers renders to its own `/etc/wireguard/wg-<class>.conf` (class interface `wg-<class>`) and is brought up independently; apply temps live in `/run/vacuum-wall/`. Rendered configs are overwritten on each apply. |
| networkd | `config/network/config.json` | `data/networkd/` | `/etc/systemd/network/99-<name>.network` | The JSON file defines per-interface static addresses, routes, DNS, DHCP, and link settings. Each entry renders to a `99-<name>.network` INI file. Stale files are cleaned on apply. Public DNS servers are auto-synced to dnsmasq upstreams. |
| ACME | `config/acme/config.json` | `data/acme/` | Certificate and key files | acme.sh manages its own state, renewal scheduling, and account keys. Vacuum Wall triggers issuance and renewal but does not maintain independent ACME state. Account registration (email, CA provider) is stored in the declarative config. |
#### Background Polling
@@ -91,19 +161,171 @@ The daemon runs background polling tasks for subsystems with external runtime st
| wireguard | 10s | Peer connections/handshakes change frequently |
| dnsmasq | 10s | Lease file + service status |
| networkd | 10s | Interface up/down, DHCP address changes |
| system | 1s | Real-time metrics (load/memory/swap/traffic) |
| nginx | 60s | Drift re-collection — the collector is a **pure re-read** of the JSON config and rendered artifacts on disk, so polling re-collects manual edits and out-of-band applies. The one-shot nginx legacy-format migration itself is *not* part of the read path — it runs once at startup via `lib/bootstrap.py` (`nginx.migrate_config_file()`). |
| acme | 300s | Drift re-collection — the collector is a **pure re-read** of acme.sh state. The only self-heal in the ACME path is `normalize_acme_home()` (reopens group access on the acme.sh tree, which acme.sh hardens to owner-only on every run). |
nginx and acme are not polled — they have no external runtime state.
All 7 state subsystems are polled. `auth` is **not** a state subsystem at all — auth data lives in the SQLite DB and is fetched on demand by Flask and the daemon, so it has no poll loop and no WS stream.
**Two-layer diff:** Each poll cycle classifies changes as:
- **Structural change** (zones added, peers removed, config changed): triggers `bump()` + broadcast `{"type": "versions", ...}` → full UI re-load
- **Volatile change only** (transfer counters, DHCP-assigned IPs): sends `{"type": "tick", "subsystems": [...]}` → lightweight per-subsystem re-fetch
- **Structural change** (zones added, peers removed, config changed): triggers `bump()` + broadcast `{"type": "versions", "subsystem": ..., "data": ...}` → daemon pushes the full subsystem data over WS; the client patches the model in place via `modelSet`
- **Volatile change only** (transfer counters, DHCP-assigned IPs): sends `{"type": "tick", "subsystem": ..., "data": ...}` → same in-place patch, without a version bump
- **No change**: silence
Volatile fields per subsystem: `wireguard` (peer transfer/handshake stats), `firewall` (DHCP-assigned IPs), `networkd` (DHCP addresses, link metrics). Defined per collector via `register_volatile()`.
Volatile fields per subsystem, defined per collector via `register_volatile()`: `system` (load/memory/swap/traffic), `firewall` (`interfaces[].ips` / `interfaces[].ipv6`), `networkd` (`interfaces[].addresses` only), `wireguard` (peer transfer/handshake stats — both the combined `status.peers[].*` and the per-class `status.classes[].peers[].*`).
Poll intervals are configurable via `VACUUM_WALL_POLL_INTERVALS` env var (`firewall:30,wireguard:10,...`).
On collector failure during a poll, no broadcast is sent (avoids noisy ticks). State data is set to `None`.
On collector failure **during a poll**, no broadcast is sent (avoids noisy ticks) and the existing state is **kept**`poll()` returns "no change" and does not clear the stored data (only `populate()` clears a subsystem's state to `None` when its collection fails). `broadcast_versions` additionally skips a `None` payload defensively (a null payload would clobber good client data — the next successful poll or mutation broadcasts the real value).
## Apply Bookkeeping and Pending Changes
Every config-backed subsystem records its last-applied state in two keys inside
its declarative JSON: `_last_applied_hash` (SHA-256 of the meta-stripped config)
and `_last_applied_config` (a snapshot of the config at apply time). The helpers
live in `lib/common.py`:
- **`stamp_applied(cfg)`** — writes both keys. Called by each subsystem's apply
handler after a successful apply.
- **`compute_pending(cfg)`** — returns `(pending, diff)`. Pending when the hash
is missing or stale; `diff` is a field-level `deep_diff()` between the
recorded snapshot and the current (meta-stripped) config.
- **`strip_apply_meta(cfg)` / `config_hash(cfg)`** — ignore the bookkeeping keys
when hashing or comparing configs.
- **`revert_to_applied(path)`** — rewrites a config file from its
`_last_applied_config` snapshot (re-stamped so the pending check reports it as
up to date); returns a reason instead when no baseline is recorded (never
applied).
The `status` handler exposes this cross-subsystem:
- **`GET /status/pending`** — aggregates pending changes per subsystem (the
firewall section additionally carries the advisory `uncovered_interfaces`
list).
- **`POST /status/apply-all`** — applies pending subsystems in dependency order
**networkd → firewall → wireguard → dnsmasq → nginx** — calling each
subsystem's apply handler. `{"force": true}` is forwarded only to the
firewall apply, where it overrides the management-lockout and
interface-coverage guards.
- **`POST /status/cancel-all`** — reverts every pending subsystem's config file
to its last-applied snapshot (subsystems with no recorded baseline are
skipped with a reason). Cancel touches only the declarative config files —
it never runs live-system commands.
## System Config Import
On daemon startup, `vacuum-walld` runs `import_all()` from `lib/system_import.py`
to reconcile live system configuration with the declarative JSON configs. This
ensures that configurations created by `scripts/install.sh` or edited manually
in system files are imported into the JSON source of truth, preventing drift.
Each subsystem import function parses the corresponding live system config and
updates the JSON config when they differ:
- **`import_dnsmasq`**: Parses `/etc/dnsmasq.d/vacuum-wall.conf` (managed
block between comment markers) → `config/dnsmasq/config.json`. Only writes
if config doesn't exist or differs.
- **`import_wireguard`**: Parses `/etc/wireguard/wg0.conf`
`config/wireguard/config.json`. Skips if configs match.
- **`import_networkd`**: Globs **all** `/etc/systemd/network/*.network` files →
`config/network/config.json`, stripping any numeric priority prefix from the
filename (`99-eth0.network``eth0`; `eth0.network``eth0`). Only
adds/updates interfaces; doesn't remove interfaces without a file (they may
be pending apply).
- **`import_nginx`**: **Skips entirely if `config/nginx/config.json` already
exists** — it only bootstraps the declarative config from rendered
`data/nginx/sites-enabled/*.conf` (vacuum-wall-managed files identified by
the `# Auto-generated by Vacuum Wall` header; `_acme-challenge.conf` is
skipped) on hosts where the JSON config is absent. When the config exists it
wins: re-parsing generated server blocks is lossy (backend references get
flattened to inline paths).
- **`import_firewall`**: Runs `sudo firewall-cmd --list-all-zones`
`config/firewall/config.json`. Only writes if no config file exists
(firewalld state always takes precedence).
All imports are **idempotent** and **non-destructive**: they only write when
configs differ, skip on failure (logged as warnings), and never abort daemon
startup. The returned list of updated subsystems is logged for debugging.
## Cross-Subsystem Sync Event Bus
When a subsystem's configuration changes, related subsystems are automatically
updated to stay consistent. An in-process event bus (`lib/sync.py`) decouples
subsystems — no handler calls into another handler's logic directly.
### How It Works
1. A mutation handler saves its config (e.g., adding a DHCP range).
2. The handler emits a `SyncEvent` on the event bus.
3. Subscribers react by updating related subsystem configs:
- **DnsToFirewallSync**: Adds `dhcp`, `dns` services to the firewall zone
for each interface serving a DHCP range. Back-propagates gateway (interface
IP) into DHCP ranges so clients receive their default route.
- **WgToFirewallSync**: Manages **per-access-class** firewall zones: for each
access class with peers, ensures a `vpn-<key>` zone exists with the class's
WireGuard interface (`wg-<key>`), masquerade enabled, and a UDP accept
rich rule on the class's `listen_port` (default 51820). Classes with
`lan_access: true` additionally get inter-zone accept rules for internal
subnets (derived from zones without masquerade); `lan_access: false`
(internet-only) classes get no internal rules. Stale class zones
(`vpn-<key>` whose class no longer has peers) have their WireGuard-created
entries cleaned up. The single `vpn`/51820 zone is managed **only as a
legacy fallback** when peers exist without an `access_class`; when
WireGuard is fully inactive all WireGuard-created entries (interface,
masquerade, `_source: wg` rules) are removed from the legacy zone.
- **FirewallToDhcpSync**: **Never deletes** DHCP ranges. Ranges whose
interface no longer belongs to any zone are kept in config and flagged
inactive (advisory warning). The only mutation is backfilling the
`gateway` (interface IP) on ranges for zones with masquerade enabled.
Zones with the `dhcp` service but no range are logged.
- **NetworkToAllSync**: Suggests DHCP ranges for static-IP interfaces without
ranges (advisory only). New network interfaces are logged/flagged but
**never added** to zones; the only mutation is removing interfaces no
longer present in the network config from the zones that still list them.
4. The handler refreshes state for the originating subsystem plus all
transitively affected subsystems.
### Guard Rails
- **Idempotency**: Each subscriber reads current state, computes desired state,
writes the diff. Running twice is safe.
- **No loops**: The event bus tracks `(subsystem, action)` per dispatch cycle.
Re-entrant emits for the same key are silently dropped.
- **Firewall-cmd separation**: Sync subscribers only write JSON config. They
do NOT call `firewall-cmd`. The user clicks "Apply" on the firewall page to
push to firewalld.
- **Error handling**: Subscriber exceptions are caught, logged as warnings,
and do NOT abort the originating handler.
### Frontend Impact
Minimal. The sync happens transparently in the backend. The "pending changes"
indicator on the firewall page will show pending when DHCP or WireGuard saves
(since sync writes JSON but does not call firewall-cmd).
## Daemon Startup Order
The daemon's `main()` (`daemon/server.py`) runs a fixed startup sequence after
the aiohttp app is listening on the Unix socket and WebSocket port:
1. **`system_import.import_all()`** — reconciles live system configs into the
declarative JSON (see System Config Import above). Runs **first** because it
must see absent config files in order to adopt live system state on first
start.
2. **`bootstrap()`** (`lib/bootstrap.py`) — creates the runtime `config/` +
`data/` directories for all subsystems and persists the one-shot nginx
legacy-format migration (`nginx.migrate_config_file()`). Idempotent. It
deliberately never creates config *files*: `get_config` reads are pure
(missing file → in-memory defaults), so files are materialized on the first
`save_config` (or by the import itself).
3. **`normalize_acme_home()`** — reopens group access on the acme.sh tree
(acme.sh hardens it to owner-only on every run); a failure here is logged,
never fatal.
4. **First `state_store.populate()`** — collects all subsystem state; the
version counter of every successfully populated subsystem is bumped so the
first WS snapshot is followed by a `versions` broadcast.
5. **`start_polling(loop)`** — spawns one background poll task per subsystem
(intervals per Background Polling above).
## Directory Structure
@@ -113,6 +335,8 @@ Config files are persistent, user-editable JSON that defines the desired state f
```
config/
├── auth/
│ └── config.json # JWT settings, WebAuthn RP configuration
├── dnsmasq/
│ └── config.json # DHCP ranges, static leases, DNS forwarding, custom records
├── firewall/
@@ -131,21 +355,26 @@ The `data/` directory holds generated files, credentials, and subsystem artifact
```
data/
├── auth.db # SQLite database: users, permissions, token_blacklist, refresh_tokens, webauthn_creds, init_sequence
├── nginx/
│ ├── .htpasswd # HTTP Basic Authentication credentials for management UI
│ ├── .htpasswd # HTTP Basic credentials for basic-authed proxy domains (created on demand; the management UI itself uses JWT only)
│ └── sites-enabled/ # Generated nginx server block .conf files (one per domain)
├── dnsmasq/
│ └── fragments/ # User-defined dnsmasq config fragments (appended verbatim)
├── firewall/
│ └── rules.json # Auto-generated firewall rule state backup
│ └── rules.json # Pre-apply firewall recovery snapshot
├── acme/ # ACME certificate files (acme.sh home)
├── logs/
│ └── vacuum-wall.log # Application log file
└── wireguard/ # WireGuard runtime artifacts
├── networkd/ # Generated 50-<name>.network files
├── networkd/ # Generated 99-<name>.network files
```
Both `config/` and `data/` reside within the project directory. The systemd service unit's `ReadWritePaths` directive grants the Flask process write access to both directories, while keeping the rest of the filesystem read-only. The `INSTALL_DIR` value is templated into the service unit at install time.
Both `config/` and `data/` reside within the project directory. The systemd service unit's `ReadWritePaths` directive grants the processes write access to these directories, while keeping the rest of the filesystem read-only. The `INSTALL_DIR` value is templated into the service unit at install time.
The daemon uses a **runtime directory** at `/run/vacuum-wall` (created by systemd `RuntimeDirectory=`) for secure temporary files during config apply. `tempfile.NamedTemporaryFile` writes to this directory before `sudo cp` moves files to their final destination, eliminating TOCTOU symlink races that would exist with `/tmp`. The directory is automatically removed on service stop.
`/run` is a fresh tmpfs at every boot, so volatile runtime paths must be recreated at startup. This is a hard requirement, not a best practice: with `ProtectSystem=strict`, namespace setup fails (`226/NAMESPACE`) and the unit crash-loops if any `ReadWritePaths=` entry does not exist when the unit spawns. Each `/run` path the daemon references therefore needs a boot-time creator: the unit's `RuntimeDirectory=vacuum-wall nginx` covers the daemon-owned directories, and the `system/tmpfiles.d/vacuum-wall.conf` spec (installed to `/etc/tmpfiles.d/`) pre-creates `/run/firewalld` and `/run/nginx.pid` at early boot via `systemd-tmpfiles-setup.service` (in practice firewalld creates it itself, and it starts before the daemon; nginx rewrites the pid file on start). `/run/nginx.pid` is additionally listed in the unit's `ReadWritePaths=`: the daemon's `nginx -t` opens the pid file for *writing*, so a read-only mount would fail every daemon-side `nginx -t` (and therefore `/nginx/apply`) with EROFS — the tmpfiles entry guarantees the file exists at spawn. `/run/sudo` is deliberately *not* in the unit's `ReadWritePaths=`: the daemon's sudo children use the NOPASSWD whitelist and never read or write sudo's session directory, so listing it only added a boot-time and restart-time failure mode (sudo removes `/run/sudo` when the last session ends).
## File System Layout
@@ -156,9 +385,15 @@ The following file system locations are used for integration with system service
| `/etc/nginx/conf.d/vacuum-wall.conf` | Include directive that pulls in `data/nginx/sites-enabled/*.conf`. | Vacuum Wall (lib/nginx.py) |
| `/etc/nginx/snippets/vacuum-wall-ssl.conf` | Shared SSL configuration snippet (protocols, ciphers, DH parameters, OCSP). Included by all HTTPS server blocks. | Vacuum Wall (lib/nginx.py) |
| `/etc/dnsmasq.d/vacuum-wall.conf` | Generated dnsmasq configuration file. Written from `config/dnsmasq/config.json`. | Vacuum Wall (lib/dnsmasq.py) |
| `/etc/wireguard/wg0.conf` | Generated WireGuard interface configuration. Written from `config/wireguard/config.json`. | Vacuum Wall (lib/wireguard.py) |
| `/etc/systemd/network/50-<name>.network` | Generated systemd-networkd drop-in files. Written from `config/network/config.json`, one per interface. | Vacuum Wall (lib/network.py) |
| `/etc/wireguard/<ifname>.conf` | Generated WireGuard interface configuration, written from `config/wireguard/config.json`. Per-class `wg-<class>.conf` in multi-interface mode; legacy single interface `wg0.conf`. | Vacuum Wall (lib/wireguard.py) |
| `/etc/systemd/network/99-<name>.network` | Generated systemd-networkd drop-in files. Written from `config/network/config.json`, one per interface. | Vacuum Wall (lib/network.py) |
| `/etc/sudoers.d/vacuum-walld` | Sudo whitelist for the daemon user. Defines all permitted privilege escalations. | Install script (rendered from Jinja2 template) |
| `/run/vacuum-wall` | Runtime directory for secure temp files during config apply (nginx, dnsmasq, wireguard, networkd). Created by systemd `RuntimeDirectory=`, removed on stop. | Daemon (systemd unit) |
| `/run/nginx` | Runtime directory referenced by the daemon's `ReadWritePaths=`; must exist at spawn. Created by systemd `RuntimeDirectory=` before namespace setup. | Daemon (systemd unit) |
| `/run/nginx.pid` | nginx pid file. Must exist at spawn **and** be writable by the daemon: its `nginx -t` opens the file for writing, so it needs both a boot-time creator (`system/tmpfiles.d/vacuum-wall.conf`) and a `ReadWritePaths=` entry (nginx rewrites it on start). | nginx / systemd-tmpfiles (early boot) |
| `/run/firewalld` | Root-owned runtime dir of firewalld. Must exist at spawn because of `ProtectSystem=strict` + `ReadWritePaths=` (see volatile-/run note above). Present while firewalld runs; also pre-created at early boot by `system/tmpfiles.d/vacuum-wall.conf`. | firewalld / systemd-tmpfiles (early boot) |
| `/run/sudo` | sudo's session directory. Present only while sudo sessions exist. **Not** in the unit's `ReadWritePaths=` (NOPASSWD sudo children never need it) — see volatile-/run note above. | sudo (created/removed on demand) |
| `data/auth.db` | SQLite database: users, permissions, token_blacklist, refresh_tokens, webauthn_creds, init_sequence. Created on first access via `get_db()`. | Auth layer (lib/db.py) |
The `/etc/nginx/conf.d/vacuum-wall.conf` include file ensures that all domain-specific configurations in `sites-enabled/` are loaded by nginx without modifying the main `nginx.conf`. The SSL snippet keeps TLS settings consistent across all managed domains and allows global updates from a single location.
@@ -169,25 +404,44 @@ The web UI is a single-page application built on **Hoover**, a custom lightweigh
### Request Flow (Frontend)
```
Client requests index.html ──→ nginx ──→ Flask (server-side __WS_URL_PLACEHOLDER__ substitution)
Client loads app.js ──→ Hoover initializes, mounts #sidebar and #main render roots
Hoover connects WebSocket ──→ daemon/ws (127.0.0.1:9091)
Client requests / ──→ nginx ──→ Flask (serves index.html)
Client loads /static/app.js ──→ served by nginx directly from disk (mgmt `location /static/` alias, no Flask round-trip) ──→ Hoover initializes, auth model 'check' fetch action (GET /api/auth/session; not-ok with a stored refresh token → exactly one refresh) → if no valid session, render #login
Authenticated ──→ mounts #sidebar and #main render roots
apiFetch() ──→ injects BOTH Authorization: Bearer <token> and X-Session-Id headers ──→ Flask REST API
Flask before_request ──→ validates JWT + session ID from headers, checks blacklist, verifies permissions
Hoover connects WebSocket ──→ daemon/ws (raw JWT as Sec-WebSocket-Protocol subprotocol name; legacy `Bearer <token>` subprotocol + X-Auth-Token header fallbacks accepted)
Page navigate (hash change) ──→ reactive router state updates ──→ render engine re-executes ──→ VDOM diff patches DOM
User action (form submit) ──→ apiFetch() ──→ Flask REST API ──→ daemon/client.py ──→ vacuum-walld
WebSocket message (versions) ──→ topic match ──→ page load() re-executed ──→ state updated ──→ render engine patches DOM
Token expiry ──→ TTL-driven scheduleRefresh() (timer at access-token TTL 60s, min 30s) ──→ POST /api/auth/refresh ──→ new tokens
WS connect ──→ snapshot (full state) / versions + tick deltas (per-subsystem data) ──→ modelSet() patches model in place ──→ render engine VDOM-diffs and patches only changed DOM nodes
WS close ×3 ──→ refreshAuth() + reconnect; 2 consecutive failed refresh+reconnect episodes ──→ give up: no more reconnect attempts until the page is reloaded (REST API keeps working)
```
The SPA entry point only serves `index.html` at `/`. All other paths return 404. Non-API, non-static paths are not served by Flask — the client-side router handles all navigation via hash changes. A dedicated `/vendor/<path>` route serves vendored JS libraries. On the management domain, nginx serves `/static/` directly from `webui/static/` via a `location /static/` alias in the generated server block, so asset requests never reach Flask in production; Flask's static route remains as the dev-mode fallback.
### Component Model
Each route is a `definePage()` component with reactive state, async data loading, and WebSocket auto-refresh. Pages are mounted using `hComp(page, key)` in the router, where the key determines lifecycle boundaries. The same key reuses the component instance (preserving state); a different key unmounts the old page and mounts the new one.
### No Build Step
All JavaScript is served as ES modules. The `?v=N` query string param version-pins asset imports for cache invalidation. Dev mode (`VACUUM_WALL_DEV`) disables aggressive static asset caching.
All JavaScript is served as ES modules. Cache invalidation is handled via HTTP cache-control headers: the management domain's `/static/` assets carry `Cache-Control: no-cache` (browsers revalidate every load; unchanged files return 304 via nginx's built-in ETag), so updates are picked up on the next page load. Dev mode (`VACUUM_WALL_DEV`) uses short TTLs instead.
### WebSocket Broadcast
Because there is no build step, backend changes can be hot-reloaded too: the `vacuum-wall` systemd unit defines `ExecReload=` which sends SIGHUP to the Flask process — Flask auto-reloads its `webui.*` and `lib.*` modules and then restarts itself, so `systemctl reload vacuum-wall` picks up code changes without a full stop/start.
The daemon broadcasts state-change notifications via WebSocket. Hoover's `subscribe` mechanism maps page-level topic subscriptions to automatic `load()` re-executions. Messages are debounced (300ms) and in-flight loads are aborted before re-loading, ensuring the UI always displays the latest available data.
### WebSocket Data Streaming
The daemon pushes state over the WebSocket — no HTTP round-trip for auto-refresh. On connect, after the JWT handshake, it sends a full snapshot (`{"type": "snapshot", "data": {subsystem: state|null, …}}`). On every structural change it broadcasts a per-subsystem delta (`{"type": "versions", "subsystem": …, "data": …}`); on volatile-only changes it sends `{"type": "tick", "subsystem": …, "data": …}`. The client's `handleMessage` patches the matching reactive model in place via `modelSet()`, and the VDOM diff touches only the changed nodes. HTTP remains the fallback for the initial load (3s timer) and for reconnect recovery.
## Firewall Interface-Coverage Invariant
A core apply-path guarantee: every network-managed interface (`lo` and `wg*` excluded) must be covered by a zone in `config/firewall/config.json` **or** listed under the top-level `unmanaged` key. The config is the source of truth for zone interfaces — an omitted `interfaces` key counts as an empty list, so there are no hands-off zones.
The check is the pure `lib.firewall.validate_coverage()`, enforced at:
- **Save time**`POST`/`PATCH /firewall/config` returns 400 when the proposed config would leave an interface uncovered.
- **Apply time**`POST /firewall/config/apply` returns 409 on coverage failure; `{"force": true}` (e.g. from the status apply-all endpoint) overrides the guard.
- **Live drift is advisory only** — the firewall state carries an `uncovered_interfaces` field and the status pending summary surfaces it, but live coverage never blocks a save or apply on its own.
## Zone Model
+298 -100
View File
@@ -48,7 +48,7 @@ This file defines all DHCP server settings and DNS resolution behavior for the d
| Field | Type | Required | Description |
|---|---|---|---|
| `ranges` | array | No | One or more DHCP address pools. Each range defines a subnet from which addresses are leased. Default: `[]`. |
| `ranges[].interface` | string | Yes | Network interface on which to serve this DHCP range (e.g., `eth1`). |
| `ranges[].interface` | string | No | Network interface on which to serve this DHCP range (e.g., `eth1`). Omit for a global range served on all interfaces (renders an untagged `dhcp-range`). |
| `ranges[].start` | string | Yes | First IP address in the pool. |
| `ranges[].end` | string | Yes | Last IP address in the pool. |
| `ranges[].lease_time` | string | No | DHCP lease duration. Accepts values like `12h`, `1d`, `30m`. Default: `12h`. |
@@ -56,7 +56,7 @@ This file defines all DHCP server settings and DNS resolution behavior for the d
| `ranges[].dns` | string | No | DNS server address advertised to DHCP clients. Typically the Vacuum Wall host's LAN IP. |
| `static_leases` | array | No | Fixed IP assignments tied to MAC addresses. Clients with matching MACs always receive the specified IP. Default: `[]`. |
| `static_leases[].mac` | string | Yes | MAC address of the client (colon-separated lowercase hex). |
| `static_leases[].ip` | string | Yes | The IP address to assign to this MAC. Must be outside the dynamic pool ranges. |
| `static_leases[].ip` | string | Yes | The IP address to assign to this MAC. Vacuum Wall does not validate that this is outside the dynamic pool ranges — keep it outside the pool to avoid address conflicts. |
| `static_leases[].hostname` | string | No | Hostname to associate with the lease. Used for reverse DNS and mDNS. |
### DNS Fields
@@ -76,42 +76,15 @@ Additional dnsmasq directives can be appended verbatim by placing plain-text fil
**File**: `config/nginx/config.json`
This file defines reverse proxy domains with path-based routing, and global SSL settings. The application renders it into per-domain server block files in `data/nginx/sites-enabled/` and into the shared SSL snippet at `/etc/nginx/snippets/vacuum-wall-ssl.conf`.
This file defines named backends (path-based routing definitions), reverse proxy domains that reference those backends, and global SSL settings. The application renders it into per-domain server block files in `data/nginx/sites-enabled/`, the include file `/etc/nginx/conf.d/vacuum-wall.conf` (which also defines the `$connection_upgrade` map used for WebSocket pass-through), the shared SSL snippet at `/etc/nginx/snippets/vacuum-wall-ssl.conf`, and a catch-all ACME challenge site at `data/nginx/sites-enabled/_acme-challenge.conf` (a port-80 `default_server` serving `/.well-known/acme-challenge/` from the `data/acme/www` webroot for domains without a dedicated server block yet).
```json
{
"domains": {
"app.example.com": {
"force_ssl": true,
"cert": "acme",
"auth": {
"user": "admin",
"htpasswd": "/home/wall/vacuum-wall/data/nginx/.htpasswd"
},
"paths": {
"/": {
"backend": {
"host": "192.168.2.50",
"port": 8080,
"proto": "http"
},
"headers": {
"X-Forwarded-Proto": "https"
}
},
"/api": {
"backend": {
"host": "192.168.2.51",
"port": 3000,
"proto": "http"
},
"auth": null
}
}
},
"mgmt.example.com": {
"force_ssl": true,
"cert": "acme",
"backends": {
"webui": {
"label": "Vacuum Wall WebUI",
"builtin": true,
"_migrated": true,
"paths": {
"/": {
"backend": {
@@ -120,10 +93,7 @@ This file defines reverse proxy domains with path-based routing, and global SSL
"proto": "http"
},
"is_management": true,
"auth": {
"user": "admin",
"htpasswd": "/home/wall/vacuum-wall/data/nginx/.htpasswd"
}
"auth": null
},
"/ws": {
"backend": {
@@ -134,6 +104,37 @@ This file defines reverse proxy domains with path-based routing, and global SSL
"is_websocket": true
}
}
},
"nas": {
"label": "NAS",
"paths": {
"/": {
"backend": {
"host": "192.168.2.50",
"port": 8080,
"proto": "http"
},
"headers": {
"X-Forwarded-Proto": "https"
}
}
},
"auth": {
"user": "admin",
"htpasswd": "data/nginx/.htpasswd"
}
}
},
"domains": {
"app.example.com": {
"force_ssl": true,
"cert": "acme",
"backend": "nas"
},
"mgmt.example.com": {
"force_ssl": true,
"cert": "acme",
"backend": "webui"
}
},
"ssl": {
@@ -144,38 +145,58 @@ This file defines reverse proxy domains with path-based routing, and global SSL
}
```
### Domain Entries
### Backends
The `domains` object maps domain names (keys) to proxy configurations. Each entry produces a separate nginx `server` block. All routing is path-based — a domain can proxy multiple paths to different backends.
The `backends` object maps backend names (keys) to shared routing definitions. Each backend carries the path map and an optional auth block; domains reference a backend by name and serve all of the backend's paths. Paths live on the backend — a domain entry never carries inline `paths`.
| Field | Type | Required | Description |
|---|---|---|---|
| `paths` | object | Yes | Path-to-config map. Each key is a URL path (e.g., `"/"`, `"/api"`). No catch-all unless `"/"` is explicitly defined. |
| `force_ssl` | boolean | No | Enable HTTPS redirect. HTTP requests to this domain receive a 301 redirect to HTTPS. Default: `true`. |
| `cert` | string | No | Certificate provisioning method. One of: `"acme"`, `"file"`, or `"selfsigned"`. Omit for domains that don't need a dedicated cert. |
| `auth` | object | No | Domain-level HTTP basic auth configuration (`{ user, htppasswd }`). Applies to all paths unless overridden at the path level. |
| `label` | string | Yes (on create) | Human-readable display name for the backend. Required when adding via `POST /nginx/backends/add`. |
| `paths` | object | Yes | Path-to-config map (schema in [Path Entries](#path-entries) below). |
| `auth` | object | No | Backend-level HTTP basic auth (`{ user, htpasswd }`). Used by any domain referencing this backend unless overridden at the domain level. |
| `builtin` | boolean | No (read-only) | Read-only flag set on the built-in `webui` backend. Builtin backends cannot be modified or removed. |
| `_migrated` | boolean | No (internal) | Internal marker set by the legacy-format migration. Not user-settable; stripped from API responses. |
Backends are managed through the daemon endpoints `GET /nginx/backends` (secrets stripped; each entry reports a `has_auth` boolean instead of the auth object), `PATCH /nginx/backends` (deep-merge partial update; `auth: null` or `auth: false` removes auth), `POST /nginx/backends/add` (creates a new backend; `400` if the name already exists), and `DELETE /nginx/backends/remove` (`400` for builtin backends, `409` when a domain still references the backend).
### Path Entries
Each entry under `paths` defines a location block and its proxy backend.
Each entry in a backend's `paths` map defines an nginx `location` block and its proxy target.
| Field | Type | Required | Description |
|---|---|---|---|
| `backend` | object | Yes | The upstream service for this path. |
| `backend.host` | string | Yes | IP address or hostname of the backend service. |
| `backend.port` | integer | Yes | Port the backend service is listening on. |
| `backend.proto` | string | No | Protocol: `http` or `https`. Default: `http`. |
| `headers` | object | No | Custom proxy headers (key-value pairs). Supports nginx variable interpolation. |
| `auth` | object \| null | No | Path-level auth override. `{ user, htppasswd }` replaces domain-level auth. `null` disables auth for this path. |
| `is_management` | boolean | No | Marks this path as the Vacuum Wall WebUI backend. Suppresses security headers (X-Frame-Options, etc.) so the SPA works correctly. |
| `backend.proto` | string | Yes | Protocol: `http` or `https`. Required — no default; absence is a validation error when adding or updating a backend. |
| `headers` | object | No | Custom proxy headers (key-value pairs). Supports nginx variable interpolation. Not rendered on `is_management` paths. |
| `auth` | object \| null | No | Path-level auth override. `{ user, htpasswd }` replaces domain/backend-level auth. `null` renders `auth_basic off` for this path. |
| `is_management` | boolean | No | Marks this path as the Vacuum Wall WebUI backend. The server block gets a `/static/` alias block serving `webui/static/` from disk (with `no-cache` revalidation), uses the dedicated `wall_mgmt_access.log` / `wall_mgmt_error.log` log files, and suppresses security headers (X-Frame-Options, etc.) so the SPA works correctly. |
| `is_websocket` | boolean | No | Marks this path as a WebSocket pass-through. Disables auth, sets Upgrade/Connection headers, uses extended timeouts. |
### Domain Entries
The `domains` object maps domain names (keys) to proxy configurations. Each entry produces a separate nginx `server` block and references a shared backend by name — all of that backend's paths are served under the domain.
| Field | Type | Required | Description |
|---|---|---|---|
| `backend` | string | Yes | Name of the backend (in `backends`) to proxy through (e.g., `"webui"`). Must reference an existing backend. |
| `force_ssl` | boolean | No | Enable HTTPS redirect. HTTP requests to this domain receive a 301 redirect to HTTPS. Default: `true`. |
| `cert` | string \| object | No | Certificate provisioning method. One of: `"acme"`, `"file"`, or `"selfsigned"`. Omit for domains that don't need a dedicated cert. |
| `cert_path` | string | No | For `cert: "file"`: path to the certificate file. (Also settable as `cert: { "cert_path": ..., "cert_key_path": ... }` in dict form.) |
| `cert_key_path` | string | No | For `cert: "file"`: path to the private key file. |
| `auth` | object \| null | No | Domain-level HTTP basic auth override (`{ user, htpasswd }`). Takes precedence over the referenced backend's `auth`; see [Auth Inheritance Rules](#auth-inheritance-rules). |
### Auth Inheritance Rules
- Domain-level `auth` applies to all paths unless overridden.
Effective auth for a domain is resolved in order: **domain `auth` → referenced backend `auth` → `None`**.
- A domain `auth: { ... }` overrides the referenced backend's auth for that domain; a domain without an `auth` key falls back to the backend's.
- Path-level `auth: null` means "no auth" for that path.
- Path-level `auth: { ... }` overrides domain-level for that path.
- No other domain-level settings inherit — `headers` is path-only.
- Path-level `auth: { ... }` overrides for that path.
- No other settings inherit between backends and domains `headers` is path-only.
**API auth form.** When adding or updating a domain through the API, `auth` may be given as `{ user, pass }`. The daemon writes the password into the `.htpasswd` file (SHA-256 crypt, default `data/nginx/.htpasswd`, or the `htpasswd` path supplied in the auth object) and persists only `{ user, htpasswd }` — the raw password is never stored in the config.
### Path ordering
@@ -188,12 +209,14 @@ The `cert` field is a string that selects the provisioning method:
| Value | Description |
|---|---|
| `acme` | Vacuum Wall uses acme.sh to request and renew an ACME certificate via the HTTP-01 challenge. The nginx configuration serves ACME challenge files at `/.well-known/acme-challenge/`. |
| `file` | Use a pre-existing certificate and private key from the local file system. Vacuum Wall will not attempt to renew these certificates. |
| `file` | Use a pre-existing certificate and private key from the local file system, via the domain's `cert_path` / `cert_key_path` fields (or the dict form `cert: { "cert_path": ..., "cert_key_path": ... }`). Vacuum Wall will not attempt to renew these certificates. |
| `selfsigned` | Vacuum Wall generates a self-signed certificate and private key on first apply. Useful for internal domains or testing. The generated certificate is stored at `data/certs/`. |
### Management Domain
The Vacuum Wall admin interface is configured as a regular domain entry under `domains`, with `is_management: true` on the path pointing to the Flask app. A second path (`/ws`) with `is_websocket: true` provides WebSocket pass-through for real-time state updates. This replaces the legacy `management` top-level key.
The Vacuum Wall admin interface is configured as a regular domain entry under `domains` that references the built-in `webui` backend (`"backend": "webui"`). That backend carries `is_management: true` on the root path (Flask app) and a `/ws` path with `is_websocket: true` for WebSocket pass-through. Because the built-in `webui` backend's root path has `auth: null`, the management path never gets nginx basic auth — management authentication is the Flask-layer JWT (bearer tokens); nginx `auth_basic` would suppress the SPA's Bearer requests. This replaces the legacy inline-`paths` form in which the management domain carried its own root and `/ws` paths (see [Backward Compatibility](#backward-compatibility)).
For a management domain without an explicit `cert` (or with `cert: "selfsigned"`), the apply step auto-generates a self-signed certificate at `data/certs/<domain>.crt` / `data/certs/<domain>.key` (RSA-2048, 365 days) if one is not already present.
The application can create the `.htpasswd` file programmatically via `write_htpasswd()` (using passlib's SHA-256 crypt). Manual creation is also possible:
@@ -203,9 +226,11 @@ htpasswd -bc data/nginx/.htpasswd admin yourpassword
### Backward Compatibility
Config files using the legacy format are auto-migrated on first load:
- Domain entries with a top-level `backend` key are wrapped into `paths["/"]`.
- A legacy `management` top-level key is migrated into `domains[management.domain]` with `is_management` on the root path and a `/ws` WebSocket path.
Config files using the legacy format are auto-migrated. The migration runs in-memory on every config read and is persisted to disk one-shot at daemon startup. It performs three steps:
1. Materializes the builtin `webui` backend (marked `_migrated: true`). The daemon handler's migration pass additionally harvests the legacy management domain's root-path auth into `backends.webui.auth`.
2. Rewrites legacy management domains (a root path pointing at `127.0.0.1:9090` with `is_management` and a `/ws` path pointing at `127.0.0.1:9091` with `is_websocket`) to `"backend": "webui"`, deleting their inline `paths` and `auth`.
3. Strips the legacy `application: "webui"` key.
### Global SSL Settings
@@ -235,16 +260,16 @@ This file stores the ACME account settings used by acme.sh for certificate provi
| Field | Type | Required | Description |
|---|---|---|---|
| `email` | string | No | Contact email for the ACME account. Used for certificate expiry notifications and recovery. Populated automatically when an account is registered via the WebUI. Default: `""`. |
| `ca` | string | No | ACME CA provider. One of: `"letsencrypt"` (Let's Encrypt), `"zerossl"` (ZeroSSL). Populated automatically when an account is registered. Default: `""`. |
| `ca` | string | No | ACME CA server. Any `server` string — passed through verbatim to `acme.sh --server` (e.g., `letsencrypt`, `zerossl`, or a private/staging CA). Not a closed enum. Populated automatically when an account is registered (the WebUI defaults to `letsencrypt` when no server is given). Default: `""`. |
### Account Registration
ACME account registration is handled entirely through the WebUI. When the user registers an account:
1. The user navigates to the Certificates page and clicks "Register Account".
2. Provides an email address and selects a CA provider (Let's Encrypt or ZeroSSL).
3. The backend calls `acme.sh --register-account` with the provided parameters.
4. On success, the `email` and `ca` fields in `config/acme/config.json` are populated, and acme.sh writes its `.account.conf` file under `data/acme/`.
2. Provides an email address and a CA server (defaults to `letsencrypt`).
3. The backend calls `acme.sh --register-account -m <email> --server <ca>`.
4. On success, the `email` and `ca` fields in `config/acme/config.json` are populated, and acme.sh writes its account state under `data/acme/` (modern acme.sh v3.x writes `account.conf`, without a leading dot).
Before any certificate can be issued, an ACME account must be registered. The certificate validation flow includes a blocking check (`account_registered`) that prevents issuance if no account exists.
@@ -252,66 +277,133 @@ Before any certificate can be issued, an ACME account must be registered. The ce
After registration, the account can be managed from the WebUI:
- **Update email**: The Settings modal allows changing the contact email, which triggers an update via `acme.sh --register-account -u`.
- **Update email**: The Settings modal allows changing the contact email, which triggers an update via `acme.sh --register-account -m <email>` (there is no `-u` flag; re-running account registration with the new email updates the account).
- **Deactivate account**: The Settings modal includes a button to deactivate the account via `acme.sh --deactivate-account`, which clears the `email` and `ca` fields and removes the ACME account.
### ACME Home Directory
acme.sh stores its state under `data/acme/` (the ACME home directory). Key files:
- `.account.conf` — ACME account credentials and settings (contains `ACME_LEEMAIL`, `ACME_MCA`).
- `<domain>/` — Per-domain certificate and key files issued by acme.sh.
- `account.conf` — ACME account credentials and settings (contains `ACME_LEEMAIL`, `ACME_MCA`). Modern acme.sh (v3.x) writes `account.conf` (no leading dot); older v2.x wrote `.account.conf`, and both names are still recognized.
- `ca/<server>/` — Per-CA account files, keyed by the ACME server name (e.g., `ca/letsencrypt/`).
- `<domain>/` — Per-domain certificate and key files issued by acme.sh. For ECC certificates the directory is `<domain>_ecc/`; `find_cert_dir()` checks the `_ecc` directory first, then the plain `<domain>/` directory.
- `www/` — ACME HTTP-01 webroot. Challenge files are served from here by nginx.
The application reads `.account.conf` to determine registration status. If the file is missing or lacks required keys, the account is considered unregistered.
The application determines registration status in this order: `account.conf` `.account.conf` → the declarative `config/acme/config.json` (kept in sync by the register/email handlers). If no source yields both an email and a CA, the account is considered unregistered.
## ACME Configuration
In addition to ACME-issued certificates, `POST /acme/self-signed` (daemon endpoint) generates a self-signed certificate for a domain under `data/certs/` (takes a `days` parameter, default `365`; idempotent — skips generation when the cert and key already exist).
**File**: `config/acme/config.json`
## Auth Configuration
This file stores the ACME account settings used by vacuum-wall for automatic certificate issuance via acme.sh. Account registration is performed exclusively through the WebUI — the Certificates page provides a "Register Account" modal where the user enters an email and selects a CA provider.
**File**: `config/auth/config.json`
This file defines JWT settings and WebAuthn Relying Party configuration for the authentication system.
```json
{
"email": "",
"ca": ""
"jwt": {
"access_token_ttl": 900,
"refresh_token_ttl": 604800,
"algorithm": "HS256"
},
"webauthn": {
"enabled": true,
"rp_name": "Vacuum Wall"
}
}
```
### ACME Config Fields
### JWT Fields
| Field | Type | Required | Description |
|---|---|---|---|
| `email` | string | No (WebUI) | Contact email for the ACME account, used for certificate expiry notifications and recovery. Populated when the user registers an account via the WebUI. Default: `""`. |
| `ca` | string | No (defaults to `letsencrypt`) | ACME CA provider. One of: `"letsencrypt"`, `"zerossl"`. Populated during account registration. Default: `""` (acme.sh defaults to Let's Encrypt if omitted). |
| `access_token_ttl` | integer | No | Access token lifetime in seconds. Code fallback default: `900` s; the fresh-install bootstrap writes `300` s (5 min). |
| `refresh_token_ttl` | integer | No | Refresh token lifetime in seconds. Default: `604800` (7 days). |
| `algorithm` | string | No | JWT signing algorithm. Default: `"HS256"`. |
### Account Registration Flow
**Note:** JWT signing secrets are per-user, not shared. Each user's secret is auto-generated as a 32-byte base64url token (`secrets.token_urlsafe(32)`) and stored in the `users.jwt_secret` database column. Secrets are rotated on password change to invalidate all prior sessions.
1. User navigates to the Certificates page in the WebUI.
2. Clicks "Register Account" and provides an email address, optionally selecting a CA provider.
3. The application calls `acme.sh --register-account -m <email> --server <ca>` as a privileged operation via the daemon.
4. On success, `config/acme/config.json` is updated with the email and CA. acme.sh writes its own state to `data/acme/.account.conf`.
5. The `account_registered` check in the certificate validation pipeline transitions from blocking to passing, enabling certificate issuance.
### WebAuthn Fields
The `account_registered` check is **blocking** — certificate issuance and validation will fail until an ACME account is registered. The `email_configured` check is **non-blocking** — it produces a warning if the email is empty but does not prevent issuance.
The WebAuthn config block holds only two fields:
### Account Management API
| Field | Type | Required | Description |
|---|---|---|---|
| `enabled` | boolean | No | Whether WebAuthn is enabled. Default: `true`. |
| `rp_name` | string | No | Display name for the WebAuthn Relying Party. Shown during credential registration. Default: `"Vacuum Wall"`. |
| Endpoint | Method | Description |
`rp_id` and `origin` are **not** config fields. They are derived per-request from the management domain the request arrives on and validated against the live management domains (the WebAuthn endpoints refuse domains that do not serve the management UI).
## Database Schema
The SQLite database at `data/auth.db` stores authentication data across six tables. Created automatically on first access via `get_db()`.
### users
| Column | Type | Description |
|---|---|---|
| `/api/certs/account` | `GET` | Returns account status: registered, email, CA provider. |
| `/api/certs/account/register` | `POST` | Registers a new ACME account. Body: `{ "email": "..." }`. Optional: `{ "server": "letsencrypt" }`. |
| `/api/certs/account` | `DELETE` | Deactivates the ACME account via `acme.sh --deactivate-account`. Clears email and CA from config. |
| `/api/certs/email` | `POST` | Updates the contact email on an existing account. Body: `{ "email": "..." }`. |
| `id` | INTEGER | Auto-increment primary key |
| `username` | TEXT | Unique username |
| `password_hash` | TEXT | Argon2id password hash |
| `jwt_secret` | TEXT | Per-user JWT signing secret (32-byte base64url) |
| `created_at` | INTEGER | Unix timestamp (auto-set) |
### ACME Home Directory
### permissions
acme.sh stores its operational state under `data/acme/`. The application reads `data/acme/.account.conf` to determine whether an account is registered. Required keys: `ACME_LEEMAIL` and `ACME_MCA`. Their absence or the file's absence means the account is unregistered.
| Column | Type | Description |
|---|---|---|
| `id` | INTEGER | Auto-increment primary key |
| `username` | TEXT | Foreign key to `users.username` (CASCADE on delete) |
| `subsystem` | TEXT | Subsystem name (e.g., `"firewall"`, `"dhcp"`, `"auth"`) |
| `level` | TEXT | Permission level: `"read"` or `"rw"` |
UNIQUE constraint on `(username, subsystem)`.
### token_blacklist
| Column | Type | Description |
|---|---|---|
| `jti` | TEXT | Primary key — JWT unique identifier |
| `token_type` | TEXT | `"access"` or `"refresh"` |
| `expires` | INTEGER | Unix timestamp of token expiry |
Used to invalidate tokens on logout and password change. Expired entries are cleaned up by the daemon's periodic poll loop (at most every 60 seconds) and probabilistically (roughly 2% of the time) inside `blacklist_token()` — not on every refresh.
### refresh_tokens
| Column | Type | Description |
|---|---|---|
| `username` | TEXT | Primary key (unique) — the owning user |
| `jti` | TEXT | JWT unique identifier of the current refresh token |
| `issued_at` | INTEGER | Unix timestamp when the refresh token was issued |
At most one active refresh session per user: the `username` column is unique, so issuing a new refresh token replaces the stored entry for that user. The active refresh token is blacklisted and removed on logout and password change.
### webauthn_creds
| Column | Type | Description |
|---|---|---|
| `id` | INTEGER | Auto-increment primary key |
| `username` | TEXT | Foreign key to `users.username` (CASCADE on delete) |
| `credential_id` | TEXT | Base64url-encoded credential ID |
| `public_key` | TEXT | Base64url-encoded public key |
| `sign_count` | INTEGER | Signature counter (replay prevention) |
| `name` | TEXT | User-assigned display name |
| `transports` | TEXT | JSON array of transport types |
UNIQUE constraint on `(username, credential_id)`.
### init_sequence
| Column | Type | Description |
|---|---|---|
| `seq` | INTEGER | Primary key — bookkeeping sequence marker |
## WireGuard Configuration
**File**: `config/wireguard/config.json`
This file defines the WireGuard server interface and all connected peers. The application renders it into `/etc/wireguard/wg0.conf` and applies it with `wg-quick`. The file is created automatically when `initialize()` generates the server key pair via `wg genkey` / `wg pubkey`.
This file defines the WireGuard server interface, access classes, and all connected peers. The application renders it into `/etc/wireguard/wg0.conf` and applies it with `wg-quick`. The file is created automatically when `initialize()` generates the server key pair and pre-seeds default access classes.
```json
{
@@ -321,9 +413,31 @@ This file defines the WireGuard server interface and all connected peers. The ap
"private_key": "<generated>",
"public_key": "<generated>",
"addresses": ["10.137.0.1/24"],
"server_endpoint": "vpn.example.com:51820",
"description": "Main WireGuard server",
"post_up": null,
"post_down": null
},
"access_classes": {
"full": {
"name": "Full LAN Access",
"description": "Peers get full access to internal networks",
"subnet": "10.137.0.0/24",
"listen_port": 51820,
"lan_access": true,
"private_key": "<generated>",
"public_key": "<generated>"
},
"internet": {
"name": "Internet Only",
"description": "Peers can only reach the internet",
"subnet": "10.137.1.0/24",
"listen_port": 51821,
"lan_access": false,
"private_key": "<generated>",
"public_key": "<generated>"
}
},
"peers": {
"alice": {
"public_key": "<auto-generated>",
@@ -331,7 +445,9 @@ This file defines the WireGuard server interface and all connected peers. The ap
"endpoint": "203.0.113.1:51820",
"allowed_ips": ["0.0.0.0/0"],
"persistent_keepalive": 25,
"preshared_key": null
"preshared_key": null,
"description": "Alice's office laptop",
"access_class": "full"
}
}
}
@@ -346,9 +462,41 @@ This file defines the WireGuard server interface and all connected peers. The ap
| `private_key` | string | Yes (after init) | Base64-encoded private key for the server interface. Generated automatically by `initialize()` via `wg genkey`. |
| `public_key` | string | Yes (after init) | Corresponding public key. Generated automatically by `initialize()` via `wg pubkey`. |
| `addresses` | array | No | IP address(es) assigned to the server interface in CIDR notation (e.g., `10.137.0.1/24`). Default: `["10.137.0.1/24"]`. |
| `server_endpoint` | string | No | External hostname:port for client connection. Used in generated client configs. Default: `""`. |
| `description` | string | No | Free-text description of the WireGuard server. Default: `""`. |
| `post_up` | string | No | Shell command to run after the interface is brought up. Common uses: adding NAT rules, enabling IP forwarding for the tunnel. Set to `null` to omit. Default: `null`. |
| `post_down` | string | No | Shell command to run after the interface is brought down. Used to clean up rules added by `post_up`. Set to `null` to omit. Default: `null`. |
### Access Classes
Access classes define categories of VPN access. Each class gets its own WireGuard interface (``wg-<key>``), firewall zone (``vpn-<key>``), subnet, and listen port. Peers are assigned to a class and their config is rendered to that class's interface. Pre-seeded with `full` and `internet` defaults on first initialization. Manageable via `GET/POST/PATCH/DELETE /api/wireguard/classes`. Per-class tunnel lifecycle: `POST /api/wireguard/classes/<key>/up`, `POST /api/wireguard/classes/<key>/down` (the down route forwards to the daemon's `DELETE /wireguard/classes/<class_key>/down`).
**Class key validation.** The class `key` (object key) must be lowercase alphanumeric — anything else is rejected (`400`). `name` defaults to the key when omitted. Creating a class whose key already exists raises `409 Conflict`. Deleting a class is refused with `409 Conflict` while any peer still references it (the response lists the offending peers).
| Field | Type | Required | Description |
|---|---|---|---|
| `name` | string | No | Human-readable display name for the class. Defaults to the class key when omitted. |
| `description` | string | No | Optional description of what access level this class provides. Default: `""`. |
| `subnet` | string | Yes | CIDR subnet for the class's WireGuard interface (e.g., ``10.137.0.0/24``). Server address is derived as ``<base>.1/<prefix>``. |
| `listen_port` | integer | Yes | UDP port for the class's WireGuard interface. Must be unique per class. |
| `lan_access` | boolean | No | When ``true``, the sync subscriber adds inter-zone accept rules for internal subnets, allowing peers to reach the LAN. When ``false``, peers can only reach the internet via masquerade. Default: ``false``. |
| `private_key` | string | Yes (auto) | Base64-encoded private key for the class's WireGuard interface. Auto-generated via ``POST /api/wireguard/classes/keys/<key>``. |
| `public_key` | string | Yes (auto) | Corresponding public key. Auto-generated with ``private_key``. |
### Multi-Interface Behavior
When peers are assigned to an access class, the daemon:
1. Renders a separate ``wg-<key>.conf`` for each class that has assigned peers.
2. Each class interface gets its own private/public key pair.
3. The sync subscriber creates a ``vpn-<key>`` firewall zone per class with masquerade enabled.
4. Classes with ``lan_access=true`` get additional inter-zone rules for internal subnets.
5. ``apply`` brings up all class interfaces independently. Per-class ``up``/``down`` endpoints control individual tunnels.
### Legacy Single-Interface Mode
When no peers are assigned to any access class, the system falls back to the legacy single-interface mode where all peers share ``wg0``.
### Peer Fields
Peers are stored in an object keyed by a human-readable identifier (e.g., `alice`, `office-laptop`). Each peer entry defines a WireGuard peer configuration. When `add_peer()` is called, the peer's key pair is auto-generated. The `private_key` is stored for client configuration generation but stripped from all API responses.
@@ -361,10 +509,14 @@ Peers are stored in an object keyed by a human-readable identifier (e.g., `alice
| `allowed_ips` | array | No | CIDR blocks that traffic from this peer is allowed to route. Default: `[]` (no routing restrictions from the server side). `["0.0.0.0/0"]` allows all traffic. `["10.137.0.0/16"]` restricts traffic to the VPN subnet. |
| `persistent_keepalive` | integer | No | Keepalive interval in seconds. `25` is recommended for peers behind NAT. Set to `0` or `null` to disable. Default: `null`. |
| `preshared_key` | string | No | Optional pre-shared key for post-quantum resistance. Use `wg genpsk` to generate. Default: `null`. |
| `description` | string | No | Optional description for the peer. The API defaults it to `""` when a peer is added via the endpoint; the lib-level `add_peer()` stores `null` when the field is omitted. |
| `access_class` | string | No | Key of the access class this peer belongs to (e.g., `"full"`, `"internet"`). `null` means unassigned. Default: `null`. |
### Client Configuration Generation
When a peer's `private_key` is set (which is the case when `add_peer()` auto-generates it), the WebUI can generate a complete WireGuard client configuration file that the user can download and import into their WireGuard client app. The generated config includes the peer's interface settings, the server as a `[Peer]` entry, and the appropriate `Endpoint` and `AllowedIPs` values. The `private_key` field is written into the client config file for download but is never returned by the API. `generate_client_conf()` computes the client IP address from the server's subnet and the peer's sorted index position.
When a peer's `private_key` is set (which is the case when `add_peer()` auto-generates it), the WebUI can generate a complete WireGuard client configuration file that the user can download and import into their WireGuard client app. The generated config includes the peer's interface settings, the server as a `[Peer]` entry, and the appropriate `Endpoint` and `AllowedIPs` values. The `private_key` field is written into the client config file for download but is never returned by the API.
`generate_client_conf()` derives the client IP address and the `Endpoint` port from the peer's **access class** when the peer is class-assigned — the class's `subnet` and `listen_port` are used, not the server interface's. For unassigned peers it falls back to the server interface's `addresses[0]` and `listen_port`. The client's host index is the peer's position in the sorted list of **all** peer keys (across every class) plus 2 (index 1 is reserved for the server).
### Applying Configuration
@@ -381,7 +533,7 @@ If the interface is already up, `wg-quick up` will reconfigure it in place witho
**File**: `config/firewall/config.json`
This file defines the declarative firewalld zone configuration. The application compares it against the live firewalld state via `_compute_pending_changes()` and applies incremental changes. Runtime state backups are stored in `data/firewall/rules.json`.
This file defines the declarative firewalld zone configuration. The application compares it against the live firewalld state via `_compute_pending_changes()` and applies incremental changes. Before every apply a **pre-apply recovery snapshot** is written to `data/firewall/rules.json`: `{timestamp, default_zone, zones, config}` where `zones` is the permanent firewalld zone view (`--list-all-zones --permanent`) and `config` is the declarative config at apply time. The permanent view is what is reproducible for manual recovery.
```json
{
@@ -406,19 +558,27 @@ This file defines the declarative firewalld zone configuration. The application
}
]
}
}
},
"unmanaged": ["eth9"]
}
```
### Top-Level Fields
| Field | Type | Required | Description |
|---|---|---|---|
| `zones` | object | Yes | Zone name → zone configuration (below). |
| `unmanaged` | array | No | Network interfaces that are deliberately **not** covered by any zone. Exempts them from the [interface-coverage invariant](#interface-coverage-invariant). Default: `[]`. |
### Zone Fields
The `zones` object maps zone names (keys) to zone configurations. Each zone corresponds to a firewalld zone applied via `firewall-cmd`.
| Field | Type | Required | Description |
|---|---|---|---|
| `interfaces` | array | No | Network interfaces assigned to this zone. Computed against live state to detect pending changes. Default: `[]`. |
| `interfaces` | array | No | Network interfaces assigned to this zone. The config is the source of truth: an omitted key counts as an empty list (apply unassigns the zone's live interfaces). Default: `[]`. |
| `services` | array | No | Firewalld services to allow in this zone (e.g., `ssh`, `https`, `dns`, `dhcp`). Default: `[]`. |
| `target` | string | No | Zone target policy. One of: `DEFAULT`, `ACCEPT`, `DROP`, `REJECT`. The code maps these to firewalld's canonical target values (`default`, `ACCEPT`, `DROP`, `REJECT`). Default: `DEFAULT`. |
| `target` | string | No | Zone target policy. `ACCEPT`, `DROP`, or `REJECT` is fully managed. When the key is **omitted** (the canonical "unmanaged" notation) or normalizes to `default` (e.g. a legacy explicit `"DEFAULT"`), the live value is **preserved** — it is not diffed and never re-set by apply (firewalld cannot set `default` back). |
| `masquerade` | boolean | No | Enable IP masquerading (NAT) for this zone. Default: `false`. |
| `forward_ports` | array | No | Port forwarding rules. Each entry has an auto-generated `id` field and the standard firewalld forward-port fields. Default: `[]`. |
| `forward_ports[].id` | string | No | Auto-generated unique identifier for the port forwarding rule. Not user-settable. |
@@ -427,17 +587,33 @@ The `zones` object maps zone names (keys) to zone configurations. Each zone corr
| `forward_ports[].toaddr` | string | No | Internal IP address to forward to. Omit for broadcast forwarding. |
| `forward_ports[].toport` | integer | No | Internal port to forward to. Omit to keep the same port. |
| `rich_rules` | array | No | Rich rule entries for advanced firewall policies. Default: `[]`. |
| `rich_rules[].id` | string | No | Auto-generated unique identifier (8-hex UUID) for the rich rule. Not user-settable; assigned when the rule is added via the API. The `DELETE /firewall/rich-rules/remove` endpoint addresses rules by this `id`. |
| `rich_rules[].rule` | string | Yes | The full firewalld rich rule string, e.g., `rule family="ipv4" source address="10.0.0.0/8" reject`. |
### Applying Firewall Configuration
The `config_pending()` function compares the declarative config in `config/firewall/config.json` against the live firewalld state returned by the daemon (via `daemon.handlers.firewall.get_state()`). It returns a diff indicating which zones have pending changes for interfaces, services, target, masquerade, forward ports, and rich rules. Zones that exist live but not in config are reported as `unmanaged_zones`.
The `config_pending()` function compares the declarative config in `config/firewall/config.json` against the live firewalld state returned by the daemon (via `daemon.handlers.firewall.get_state()`). It returns a diff indicating which zones have pending changes for interfaces, services, target, masquerade, forward ports, and rich rules. Masquerade is **skipped for the `public` zone** in both the pending diff and the apply step — the public zone's masquerade is driven by the nftables propagation step described below, so diffing it would advertise a change that never happens. Zones that exist live but not in config are reported as `unmanaged_zones`, excluding the zones firewalld ships by default (`block`, `dmz`, `drop`, `external`, `home`, `host`, `internal`, `public`, `trusted`), which are always present live and never meaningful to flag. A `target` entry is only reported when the config carries an explicit target that normalizes to something other than `default`; an omitted key or a `default`-normalizing value is unmanaged, so live target drift is neither flagged nor applied. The `interfaces` entry is reported for **every** config zone — the config is the source of truth for zone interfaces, so an omitted `interfaces` key counts as an empty list and pending changes are diffed accordingly.
Both `/api/firewall/zones/<name>/services` and `/api/firewall/config/apply` reconcile **remove-then-add** against the live zone, so anything opened outside the declarative config (e.g. directly via `firewall-cmd`) is reverted on the next apply. Service changes made through the API are persisted to `config.json` to prevent this drift.
**Management-lockout guard.** The firewalld *default zone* is the catch-all for interfaces with no explicit assignment (typically the WAN), and it carries the management plane (nginx https) plus remote recovery (ssh). Changing the default zone's service set so that **neither `https` nor `ssh`** remains raises `409 Conflict` — from `POST /firewall/zones/<name>/services` and `POST /firewall/config/apply` — before any mutation runs. Send `"force": true` in the request body to override (the UI shows a confirm dialog with this effect on the Zones page). If the default zone cannot be determined, the guard fails closed.
**Interface-coverage invariant.** Every network-subsystem-managed interface must be covered by a zone in the firewall config — otherwise all traffic (and DHCP) from that segment is dropped. Guarded interfaces are the keys of the network config's `interfaces`, excluding `lo` and `wg*` (vpn zones are managed by the WireGuard sync and `lo` is normally zoneless). Because the config is the source of truth for zone interfaces (an omitted `interfaces` key counts as empty), coverage is computed from the config **alone** via `validate_coverage()` — there is no live-state fallback and no hands-off zones. Interfaces listed in the top-level `unmanaged` key are exempt. The invariant is enforced at two points:
- **Save time**`POST /firewall/config` and `PATCH /firewall/config` reject a config that leaves a managed interface uncovered with `400 Bad Request`, before anything is written.
- **Apply time**`POST /firewall/config/apply` re-checks the (possibly stale) saved config against the current network config and raises `409 Conflict` before any mutation. A conflict here means the network config changed after the firewall config was saved (e.g. a new interface no zone covers).
Send `"force": true` in the request body to override the apply-time check (the UI offers this via the Apply dialog). Live drift — an interface that is covered by the config but not in any **live** zone — is advisory only: it is surfaced as the `uncovered_interfaces` field in firewall state (see `docs/state-model.md`), the Zones-page banner, and an advisory in `GET /api/status/pending`, and is never blocked by the invariant.
**Applied baseline.** Like the other config-backed subsystems, a successful apply records `_last_applied_hash` and `_last_applied_config` (the meta-stripped config snapshot) inside `config.json`. They are internal bookkeeping — ignored by all parsing, hashing, and UI surfaces — and let the aggregate cancel action (`POST /api/status/cancel-all`) revert this file to the last applied state. Configs that have never been applied have no baseline and are skipped by cancel.
**Public-zone masquerade propagation.** With firewalld's nftables backend, traffic leaving through the public zone hits the public zone's POSTROUTING chain, so NAT only works if the public zone itself has masquerade enabled. During apply, if any non-public zone has masquerade enabled but the public zone does not, apply propagates masquerade to the public zone (and writes it back into the config); conversely, when no non-public zone needs masquerade, apply removes it from the public zone. Consistently, the `POST /firewall/masquerade` endpoint **refuses** to enable masquerade on the `public` zone directly (enable it on `internal` or a `vpn` zone instead — the API returns an error directing you there).
## Networkd (IP Configuration)
**File**: `config/network/config.json`
This file defines static IP configuration for network interfaces managed by systemd-networkd. The application renders each interface entry into a `50-<name>.network` INI file in `data/networkd/`, which the handler copies to `/etc/systemd/network/`.
This file defines static IP configuration for network interfaces managed by systemd-networkd. The application renders each interface entry into a `99-<name>.network` INI file in `data/networkd/`, which the handler copies to `/etc/systemd/network/`.
```json
{
@@ -487,7 +663,7 @@ Each key in the `interfaces` object is an interface name (e.g., `eth0`, `eth1`,
| `dhcp` | `string` | DHCP mode: `"yes"`, `"ipv4"`, `"ipv6"`, `"no"`. Controls `[Network] DHCP=` and whether `[DHCPv4]`/`[DHCPv6]` sections are rendered. |
| `routes` | `array` | Static routes. Each dict has `destination`, `gateway`, `metric`, `table`, `type`, `scope`, `gateway_on_link`, `ipv6_preference`, `initial_congestion_window`, `initial_advertised_receive_window`, `quick_ack`, `fast_open_no_cookie`, `mtu_bytes`, `protocol`, `next_hop`, `multi_path_route`. Renders to `[Route#N]` sections. |
| `link` | `object` | Link settings: `mtu_bytes`, `mac_address`, `arp`, `multicast`, `all_multicast`, `promiscuous`, `unmanaged`, `activation_policy`, `required_for_online`. Renders to `[Link]` section. |
| `dhcp_client` | `object` | DHCP client settings. Shared keys for both `[DHCPv4]` and `[DHCPv6]`: `hostname`, `duid_type`, `duid_raw_data`, `iaid`, `client_identifier`, `rapid_commit`, `anonymize`, `use_dns`, `use_ntp`, `use_sip`, `use_captive_portal`, `use_mtu`, `use_hostname`, `use_domains`, `use_routes`, `route_metric`, `send_decline`, `net_label`, `nft_set`, `ip_service_type`, `socket_priority`, `bootp`, `label`, `max_attempts`, `listen_port`, `server_port`, `mud_url`, `boot_filename`, `send_option`, `send_vendor_option`, `user_class`, `vendor_class_identifier`, `request_options`. |
| `dhcp_client` | `object` | DHCP client settings. `[DHCPv4]` and `[DHCPv6]` have **different** key sets (which sections render is controlled by `dhcp`). Shared by both: `hostname`, `duid`, `duid_type`, `duid_raw_data`, `iaid`, `anonymize`, `rapid_commit`, `use_dns`, `use_ntp`, `use_sip`, `use_captive_portal`, `use_hostname`, `use_domains`, `net_label`, `nft_set`, `send_option`, `send_vendor_option`, `user_class`. IPv4-only (`[DHCPv4]`): `client_identifier`, `use_mtu`, `use_routes`, `route_metric`, `send_decline`, `ip_service_type`, `socket_priority`, `bootp`, `label`, `max_attempts`, `listen_port`, `server_port`, `mud_url`, `boot_filename`, `vendor_class_identifier`, `request_options`. IPv6-only (`[DHCPv6]`): `send_hostname`, `prefix_delegation_hint`, `unassigned_subnet_policy`, `use_address`, `use_delegated_prefix`, `use_dnr`, `send_release`, `without_ra`, `vendor_class` (a list; each entry renders a `VendorClass=` line). |
| `bind_carrier` | `array` | Carrier interfaces to bind to. |
| `ignore_carrier_loss` | `boolean` | Ignore carrier loss events. |
| `keep_configuration` | `boolean` | Keep configuration on stop. |
@@ -521,4 +697,26 @@ When `POST /api/network/apply` is called, the handler automatically collects pub
### Generated Files
Each interface config entry produces a `50-<name>.network` file in `data/networkd/`. During apply, these are copied to `/etc/systemd/network/` and stale files (not matching any config entry) are removed. File generation uses the `systemd.syntax(7)` naming convention: first section is bare (`[Address]`, `[Route]`), subsequent sections use `#` suffix (`[Address#1]`, `[Route#2]`).
Each interface config entry produces a `99-<name>.network` file in `data/networkd/`. During apply, these are copied to `/etc/systemd/network/` and stale files (not matching any config entry) are removed. File generation uses the `systemd.syntax(7)` naming convention: first section is bare (`[Address]`, `[Route]`), subsequent sections use `#` suffix (`[Address#1]`, `[Route#2]`).
## Cross-Subsystem Dependencies
Some subsystems depend on each other. When you modify one, related subsystems
are updated automatically through the event bus.
| Trigger Subsystem | Affected Subsystem | What Happens |
|---|---|---|
| dnsmasq (DHCP range) | firewall | Zone gains `dhcp`/`dns` services. Removing the last range removes them. DHCP ranges also back-propagate gateway (interface IP) so clients receive their default route. |
| wireguard (peer add/remove) | firewall | Per-class `vpn-<key>` zones are created with `wg-<key>` interface, masquerade, UDP port rule, and inter-zone accept rules (only when ``lan_access=true``). Falls back to single `vpn` zone in legacy mode. Cleanup removes stale rules when classes have no peers. |
| firewall (zone changes) | dnsmasq | Stale DHCP ranges (for interfaces no longer in any zone) are **kept in the config and flagged inactive — never removed**. Masquerade-enabled zones ensure DHCP ranges carry the gateway. Zones with dhcp service but no range are logged as warnings. |
| network (interface config) | firewall | Zone interface assignments in firewall config are updated — new interfaces are flagged, stale ones removed. |
| network (interface config) | dnsmasq | Suggested DHCP ranges are logged when an interface has a static IP but no DHCP range. |
Note: The firewall "Apply" button is still needed to push config changes to
firewalld. Sync only updates the declarative JSON.
Additionally, on daemon startup, `lib/system_import.py` reconciles live system
configs (dnsmasq, wireguard, networkd, nginx, firewall) with the declarative
JSON. This prevents drift when configs were created by the install script or
edited manually in system files. Reconciliation only writes when the existing
JSON differs or is missing — no data is lost on re-run.
+102 -36
View File
@@ -25,13 +25,13 @@ Download the Vacuum Wall repository onto the target machine, then run the instal
# Production: all env vars
MGMT_DOMAIN=wall.example.com \
MGMT_PASS="strongpassword" \
./install.sh --user vacuum-wall
./scripts/install.sh --user vacuum-wall
# Dev mode: CLI flags, auto-detects repo owner
./install.sh --dev --mgmt-pass strongpassword
./scripts/install.sh --dev --mgmt-pass strongpassword
# mDNS (LAN-only, no DNS record needed)
./install.sh --mgmt-domain vacuum-wall.local --mgmt-pass strongpass
./scripts/install.sh --mgmt-domain vacuum-wall.local --mgmt-pass strongpass
```
### Options
@@ -40,9 +40,9 @@ All settings that can be passed as an environment variable also have a CLI flag
| Flag | Env Var | Required | Description |
|---|---|---|---|
| -- | `MGMT_DOMAIN` | No | Domain for the management WebUI. Defaults to `$hostname.local` (mDNS). Auto-detected from system hostname. **Errors if hostname is undetectable and this is not set.** |
| -- | `MGMT_DOMAIN` | No | Domain for the management WebUI. Auto-detected from the system hostname; defaults to `$(hostname -f \|\| hostname).local` (FQDN first, falling back to the short hostname; mDNS-served on the LAN). **Errors if the hostname is undetectable and this is not set.** |
| `--mgmt-domain` | `MGMT_DOMAIN` | No | (same as above) |
| `--mgmt-pass` | `MGMT_PASS` | Yes | Password for HTTP basic auth protecting the WebUI. |
| `--mgmt-pass` | `MGMT_PASS` | Yes | Password for the initial admin user (default: `admin`). Creates the admin user in the SQLite database with full `rw` permissions on all subsystems. |
| `--mgmt-user` | `MGMT_USER` | No | Username for WebUI access. Defaults to `admin`. |
| `--user, -u` | `USER_NAME` | Yes* | WebUI service user (created if it does not exist). Required for non-dev mode. In `--dev` mode, auto-detected from repo owner. |
| `--path, -p` | `INSTALL_DIR` | No | Install directory. Defaults to repo root. Set to deploy from a custom path (e.g., `/opt/vacuum-wall`). |
@@ -51,7 +51,7 @@ All settings that can be passed as an environment variable also have a CLI flag
| `--lan-ifaces` | `LAN_IFACES` | No | LAN interface names, comma-separated. Auto-detected from non-loopback, non-WAN interfaces. |
| `--force-venv` | — | No | Force recreation of the Python virtual environment. |
Run `./install.sh --help` for full usage.
Run `./scripts/install.sh --help` for full usage.
---
@@ -64,17 +64,17 @@ The `--dev` flag is designed for developers working in a git clone. It auto-dete
In dev mode, the ownership model preserves the developer's ability to work with the repository:
- **Project directory**: Owned by the repo owner (e.g., `wall`), group is the repo owner's primary group (e.g., `wall`). The developer retains full control — `git add`, `git commit`, editing code and config files all work normally.
- **Daemon access**: The daemon user (`vacuum-walld`) has the repo owner's primary group as its own primary group, granting read access to all project files. The project directory has the setgid bit (`g+s`) on all subdirectories, ensuring new files inherit the group.
- **Daemon access**: The daemon user (`walld`, i.e. `${USER_NAME}d`) has the repo owner's primary group as its own primary group, granting read access to all project files. The project directory has the setgid bit (`g+s`) on all subdirectories, ensuring new files inherit the group.
- **`.venv/` and `data/`**: Owned by the repo owner, group is the repo owner's primary group. The developer can run `pip install`, inspect logs, and manage runtime artifacts. The daemon reads `.venv/` (Python interpreter) and writes to `data/` (runtime files) via group permissions.
- **Daemon socket** (`data/daemon.sock`): Owned by `vacuum-walld:<group>` (mode `0660`). The repo owner accesses it via primary group membership.
- **Daemon socket** (`data/daemon.sock`): Owned by `walld:<group>` (mode `0660`). The repo owner accesses it via primary group membership.
### Running the Installer in Dev Mode
```bash
./install.sh --dev --mgmt-pass strongpassword
./scripts/install.sh --dev --mgmt-pass strongpassword
```
The script detects the repo owner (e.g., `wall`), creates the `vacuum-walld` daemon user with the repo owner's primary group, and sets up the ownership model described above.
The script detects the repo owner (e.g., `wall`), creates the `walld` daemon user (`${USER_NAME}d`) with the repo owner's primary group, and sets up the ownership model described above.
### Idempotent Re-Runs
@@ -88,58 +88,64 @@ You can deploy Vacuum Wall in a container or at any custom path. Use `--path` (o
```bash
# Docker volume mount example
./install.sh --path /app/vacuum-wall --user ww-app \
./scripts/install.sh --path /app/vacuum-wall --user ww-app \
--mgmt-domain proxy.internal --mgmt-pass strongpassword
```
The systemd service unit files and sudoers whitelist are rendered from Jinja2 templates at install time, substituting `USER_NAME` and `INSTALL_DIR`. This means no hardcoded paths remain after installation.
The systemd `.service` unit files and the sudoers whitelist are rendered from Jinja2 templates at install time, substituting `USER_NAME`, `USER_DAEMON_NAME`, `USER_GROUP`, `PROJECT_DIR`, and `ACME_HOME`. This means no hardcoded paths remain after installation.
---
## What install.sh Does
## What scripts/install.sh Does
The installer performs the following steps automatically:
- **Package installation**: Installs firewalld, nginx, dnsmasq, avahi-daemon, wireguard-tools, python3, python3-pip, jq, curl, iptables, nftables, and apache2-utils.
- **WebUI user creation**: Creates the WebUI user (from `--user`) as a system user if it does not exist.
- **Shared group**: Uses the WebUI user's primary group as the shared group between both service users.
- **Daemon user creation**: Creates `vacuum-walld` (derived from WebUI user name) — a system user with `NOPASSWD` sudo access for privileged operations. Owns the project directory and daemon socket.
- **Daemon user creation**: Creates the daemon user `${USER_NAME}d` (the literal `vacuum-walld` only when the WebUI user is `vacuum-wall`) — a system user with `NOPASSWD` sudo access for privileged operations. Owns the daemon socket and, outside `--dev` mode, the project directory (in dev mode the repo owner keeps project ownership).
- **Python venv**: Creates the Python virtual environment and installs project dependencies. Skips if already present (use `--force-venv` to recreate).
- **acme.sh installation**: Copies the vendored acme.sh client to the data directory for ACME certificate management. Skips if already installed.
- **acme.sh installation**: Fetches the vendored acme.sh (via `scripts/update-vendor.sh`) and installs it to `data/acme/acme.sh`, skipping if it is already present. Also installs the `system/acme-deploy.sh` deploy hook into `data/acme/deploy/acme-deploy.sh` (acme.sh only resolves hooks from its own deploy directory) and repairs ownership of the acme.sh runtime conf files under `data/acme/` — including `account.conf`, which is chmodded to `0640` — to the daemon user, so the first acme.sh run cannot fail on owner-only files.
- **Directory setup**: Creates config directories under `config/` for each subsystem's declarative JSON, and data directories under `data/` for generated files (nginx sites, dnsmasq fragments, firewall backup, WireGuard config).
- **Template rendering**: Renders system template files (`systemd/*.service`, `sudoers.d/`) via Jinja2, substituting `USER_NAME`, `INSTALL_DIR`, and `ACME_HOME`. Installed systemd and sudoers files contain no hardcoded values.
- **Sudoers whitelist**: Installs a restrictive sudoers file at `/etc/sudoers.d/vacuum-walld` granting the daemon user `NOPASSWD` sudo for only the specific privileged commands needed for firewall, nginx, dnsmasq, and acme.sh management. Validates syntax with `visudo -cf`.
- **System-directory ownership repair**: Checks top-level system directories (`/`, `/bin`, `/boot`, `/etc`, `/home`, `/opt`, `/root`, `/srv`, `/usr`, `/var`, …) for non-root ownership — some appliance images ship with system paths owned by a regular user, which trips systemd-tmpfiles' "unsafe path transition" check. Mis-owned top-level directories are chown'd to `root:root`; if deeper mis-ownership is detected, the installer warns with a full-repair command to run before re-running.
- **Static-asset permissions**: `chmod a+rX` on `webui/static/` (plus `a+x` up the parent directory chain) so nginx's `www-data` workers can serve the management UI's static assets directly from disk, regardless of checkout umask.
- **Template rendering**: Renders the systemd `.service` files and the sudoers whitelist via Jinja2, substituting `USER_NAME`, `USER_DAEMON_NAME`, `USER_GROUP`, `PROJECT_DIR`, and `ACME_HOME`. Installed systemd and sudoers files contain no hardcoded values.
- **Sudoers whitelist**: Installs a restrictive sudoers file at `/etc/sudoers.d/vacuum-walld` granting the daemon user `NOPASSWD` sudo for only the specific privileged commands needed: firewalld management (`firewall-cmd`), nginx (config test/reload, copying/removing the generated conf files), dnsmasq (restart, lease-file reads, fragment install), WireGuard (`wg`, `wg-quick`, installing `wg0.conf`), network interface queries (`ip -o link/addr show`), systemd-networkd (`networkctl` status/reload/reconfigure, managing `/etc/systemd/network`), sysctl writes, group-permission repair on the ACME home, and journal/log reads (`journalctl`, `cat /var/log/nginx/*`). Validates syntax with `visudo -cf`.
- **IP forwarding**: Enables `net.ipv4.ip_forward=1` in sysctl.conf and applies it at runtime, required for routing traffic between zones. Appends only if not already present.
- **Firewalld initialization**: Starts and enables firewalld. Opens HTTP, HTTPS, and SSH services on the public zone for management access.
- **Dnsmasq initialization**: Starts and enables dnsmasq for future DHCP/DNS serving on internal interfaces.
- **mDNS broadcast**: Enables and starts avahi-daemon so the appliance advertises its hostname (`<hostname>.local`) on the local network.
- **Self-signed certificate**: Generates a temporary self-signed X.509 certificate for the management domain with the correct CN and SAN, placed where acme.sh would store a real cert. Skips if a certificate already exists (preserves real ACME certs).
- **Management proxy configuration**: Configures nginx as a reverse proxy that forward-proxies to the WebUI at `127.0.0.1:9090`, with HTTP-to-HTTPS redirect, basic auth, and WebSocket upgrade support.
- **Credentials**: Generates an htpasswd file using `apache2-utils` (with a Python fallback) for the management proxy's basic auth. Updates existing file if already present.
- **Initial nginx config**: Writes `$PROJECT_DIR/config/nginx/config.json` with the management domain and auth settings pre-configured. Skips if the file already exists (preserves user-customized config).
- **Initial firewall config**: Writes `$PROJECT_DIR/config/firewall/config.json` with auto-detected WAN/LAN interfaces. Skips if the file already exists.
- **Systemd units**: Installs four units (rendered from Jinja2 templates):
- **Self-signed certificate**: Generates a temporary self-signed X.509 certificate for the management domain via `POST /acme/self-signed` (CN set to the domain), written to `data/certs/<domain>.crt` and `data/certs/<domain>.key` — not under `data/acme/`, where acme.sh stores issued certs. Idempotent: skips generation when both files already exist.
- **Management proxy configuration**: Registers the management domain via `POST /nginx/domains/update` (falling back to `POST /nginx/domains/add`) as the special built-in `webui` backend entry (cert `selfsigned`, forced SSL). The `/` → 127.0.0.1:9090 (Flask) and `/ws` → 127.0.0.1:9091 (daemon WebSocket) mapping is derived by the daemon from the built-in webui backend — it is not passed as paths config. Then applies nginx via `POST /nginx/apply`.
- **Admin user**: Creates the admin user (default username `admin`) with the password provided via `--mgmt-pass` in the SQLite database (`data/auth.db`), with `rw` permissions on all subsystems, and writes `config/auth/config.json` (JWT + WebAuthn settings) if missing. On re-run, updates the admin password if already present. The bootstrap runs with `VACUUM_WALL_SEED_BUILTIN_ADMIN=0`, suppressing the last-resort builtin admin seed so exactly one account exists on a fresh install.
- **Initial configs**: Once the daemon socket is up, the installer writes initial state over the daemon API: `POST /acme/self-signed` (management cert), the management domain plus `POST /nginx/apply`, and firewall zone assignment — `POST /firewall/zones/interfaces` (WAN interface → `public`) and `POST /firewall/zones/services` (http/https/ssh on `public`) when a WAN interface was detected, and `POST /firewall/zones/interfaces` (LAN interfaces → `internal`) when LAN interfaces were detected. These writes are not skipped; the only skip-if-exists rule applies to the auth config (see **Admin user**).
- **System config import**: On startup, the daemon reconciles any live system configurations (dnsmasq, wireguard, networkd, nginx, firewall) with the declarative JSON configs. This prevents drift when system files were edited manually.
- **Systemd units**: Installs four units — three `.service` files are rendered from Jinja2 templates; the `.timer` is installed verbatim:
- `vacuum-walld.service` — the privileged background daemon (aiohttp, daemon socket).
- `vacuum-wall.service` — the Flask WebUI backend.
- `vacuum-wall-acme.service` — the certificate renewal oneshot.
- `vacuum-wall-acme.timer` — periodic timer that triggers cert renewals.
- **Firewalld zones**: Creates initial zones:
- `internal` — trusted LAN zone with DHCP, DNS, and NTP services allowed.
- `vpn` — WireGuard tunnel zone.
- `vacuum-wall-acme.timer` — periodic timer that triggers cert renewals (no template variables).
A fifth file, `system/tmpfiles.d/vacuum-wall.conf`, is installed to `/etc/tmpfiles.d/vacuum-wall.conf` and `systemd-tmpfiles --create` is run immediately — load-bearing for the hardened unit: it provisions the volatile `/run` entries the daemon needs before `vacuum-walld` spawns (restored at every boot by `systemd-tmpfiles-setup.service`).
- **Firewalld zones**: Assigns initial zones via the daemon API (the installer does not create zones directly):
- `public` — the WAN interface is assigned here and the `http`, `https`, and `ssh` services are opened for management access (only when a WAN interface was detected).
- `internal` — the LAN interfaces are assigned here (only when LAN interfaces were detected); no services are added at install time.
- `vpn`**not** created by the installer. It is managed dynamically by `lib/sync.py` only while WireGuard peers exist (interface assignment, masquerade, and rich rules), and is cleaned up again when WireGuard is deactivated.
- **Legacy nginx config cleanup**: Removes the old nginx bootstrap configs (`/etc/nginx/conf.d/vacuum-wall-map.conf` and `/etc/nginx/conf.d/vacuum-wall-mgmt.conf`), which are replaced by the daemon-generated nginx configuration.
- **Service startup**: Enables and starts/restarts nginx, the daemon (`vacuum-walld`), the WebUI (`vacuum-wall`), and the ACME renewal timer. nginx is reloaded (or restarted) to pick up any config changes.
- **ACME account**: No account registration during install. Register the account via the WebUI after first login.
### Idempotent Re-Runs
`install.sh` is fully idempotent and safe to run multiple times. Re-running the script:
`scripts/install.sh` is fully idempotent and safe to run multiple times. Re-running the script:
- Skips the Python venv (use `--force-venv` to rebuild)
- Restarts `vacuum-walld`, `vacuum-wall`, and reloads `nginx` to pick up changes
- Preserves existing SSL certificates (skips self-signed generation if a cert exists)
- Preserves existing `config.json` files (skips initial write if file exists)
- Safely updates `htpasswd` (uses update mode instead of create mode)
- Preserves the existing auth config (`config/auth/config.json` is only written if missing — the only skip-if-exists config rule)
- Updates admin user password if changed
This makes it safe for development workflows: simply run `bash install.sh` again to update an existing installation.
This makes it safe for development workflows: simply run `bash scripts/install.sh` again to update an existing installation.
---
@@ -165,6 +171,28 @@ https://wall.example.com
Log in with the username and password you provided during installation.
### Environment Variables
| Variable | Default | Description |
|---|---|---|
| `VACUUM_WALL_DB_BACKEND` | `sqlite` | Database backend selection |
| `VACUUM_WALL_DB_PATH` | `data/auth.db` | SQLite database file path |
| `VACUUM_WALLD_SOCKET` | `data/daemon.sock` | Daemon Unix socket path (`daemon/server.py:669`) |
| `VACUUM_WALLD_WS_PORT` | `9091` | Daemon WebSocket port on `127.0.0.1` for real-time state streaming (`daemon/server.py:31`) |
| `VACUUM_WALL_POLL_INTERVALS` | built-in per-subsystem defaults | Comma-separated `subsystem:seconds` overrides for the state-poll intervals, e.g. `firewall:60,wireguard:5`; non-integer or ≤ 0 values are skipped with a warning (`daemon/server.py:34`) |
| `VACUUM_WALL_DEV` | unset (off) | Dev-mode flag: disables aggressive static-asset caching in the WebUI (`webui/server.py:86`) |
| `VACUUM_WALL_LOG_LEVEL` | `INFO` | Log level for the WebUI and daemon processes (`lib/logging.py:49`) |
| `VACUUM_WALL_EXTERNAL_IP_URL` | built-in detection | Custom URL for external-IP detection used by ACME (`daemon/handlers/acme.py:285`) |
| `VACUUM_WALL_SEED_BUILTIN_ADMIN` | `1` | Set to `0` to skip the last-resort builtin admin seed in `get_db()`; `scripts/bootstrap_auth.py` always sets this since bootstrap creates the operator user itself (`lib/db.py:307`) |
### Post-Deploy Verification
1. Confirm `config/auth/config.json` exists with JWT secret and WebAuthn RP configuration
2. Confirm `data/auth.db` exists with admin user present
3. Confirm the management server block has no server-level `auth_basic` directive — the management UI is authenticated by the Flask-layer JWT middleware, not nginx
4. Confirm `location /ws` has `auth_basic off` — the WebSocket is authenticated by the daemon via the raw-JWT `Sec-WebSocket-Protocol` subprotocol, never by nginx
5. Access the WebUI at `https://<management-domain>` — should show a login page
### Certificate Note
The initial certificate is **self-signed** and generated during installation. Your browser will show a security warning. This is expected. Once DNS is pointing to the appliance and port 80 is accessible from the internet, use the **Certs** tab in the WebUI to issue a real ACME certificate for the management domain. After issuance, go to the **Proxy** tab and click **Apply** to reload nginx with the new cert.
@@ -282,7 +310,9 @@ journalctl -u vacuum-wall --no-pager -n 50
nginx -t
```
Common causes include port conflicts (another service on port 80/443), missing dependencies, or file permission issues on `data/`.
Both units also keep journal output on disk under `/var/log/vacuum-wall/` (`LogsDirectory=vacuum-wall` on both units). Nginx writes per-domain access/error logs to `/var/log/nginx/wall_mgmt_*.log` for the management domain and `/var/log/nginx/<domain>_*.log` for each proxy domain.
Common causes include port conflicts (another service on port 80/443, 9090, or 9091 — the daemon's WebSocket port), missing dependencies, or file permission issues on `data/`.
### Firewall Rules Not Applying
@@ -322,13 +352,49 @@ Verify that:
- A DHCP range is configured for the correct interface. Check dnsmasq config at `data/dnsmasq/`.
- The firewall allows DHCP traffic on the internal zone: `firewall-cmd --zone=internal --list-services` should include `dhcp` and `dns`.
### Locked Out of WebUI
If you lose access to the admin account, you can reset the password directly via SQLite:
```bash
# Stop the services
sudo systemctl stop vacuum-wall vacuum-walld
# Reset password (replace 'newpassword' with desired password)
sqlite3 data/auth.db "UPDATE users SET password_hash='NEW_HASH_HERE' WHERE username='admin';"
```
The password hash must be an Argon2id hash. You can generate one:
```bash
python3 -c "from lib.password import hash_password; print(hash_password('newpassword'))"
```
Alternatively, use the SQLite prompt to directly inspect and modify user data:
```bash
sqlite3 data/auth.db ".tables"
sqlite3 data/auth.db "SELECT username FROM users;"
sqlite3 data/auth.db "SELECT * FROM permissions WHERE username='admin';"
```
### Database Corruption
If the SQLite database becomes corrupted:
1. Stop the services: `sudo systemctl stop vacuum-wall vacuum-walld`
2. Inspect: `sqlite3 data/auth.db "PRAGMA integrity_check;"`
3. If the file is unrecoverable, delete it (`rm data/auth.db`) and start the services: `sudo systemctl start vacuum-walld vacuum-wall`. The schema is recreated on startup; if the users table is empty, the last-resort builtin admin is seeded with a random password written to `/var/log/vacuum-wall/auth.log`.
4. Re-set the password via the WebUI, or use the SQLite steps under "Locked Out of WebUI".
### WebUI Not Accessible
1. Verify nginx is running: `systemctl status nginx`.
2. Test nginx configuration: `nginx -t`.
3. Check the management proxy configuration at `/etc/nginx/conf.d/vacuum-wall-mgmt.conf` (initial) or via the WebUI Proxy tab (after first apply).
4. Ensure the WebUI service is listening on port 9090: `ss -tlnp | grep 9090`.
5. If using the self-signed cert, confirm your browser trusts it or use the WebUI to issue a real ACME certificate.
3. Check the management proxy domain configuration via the WebUI Proxy tab, or by inspecting `config/nginx/config.json`.
4. Ensure the daemon is running and its Unix socket exists: `systemctl status vacuum-walld` and `ls -l data/daemon.sock` — the WebUI proxies every API call through this socket.
5. Ensure the WebUI service is listening on port 9090 (`ss -tlnp | grep 9090`) and the daemon's WebSocket endpoint on port 9091 (`ss -tlnp | grep 9091`).
6. If using the self-signed cert, confirm your browser trusts it or use the WebUI to issue a real ACME certificate.
---
@@ -338,7 +404,7 @@ Verify that:
|---|---|---|
| Daemon (privileged) | `vacuum-walld.service` | `daemon/` |
| WebUI backend | `vacuum-wall.service` | `webui/` |
| Reverse proxy | `nginx` | `/etc/nginx/conf.d/vacuum-wall-mgmt.conf` |
| Reverse proxy | `nginx` | `config/nginx/config.json` (via daemon) |
| Firewall | `firewalld` | Managed via WebUI and `firewall-cmd` |
| DHCP/DNS | `dnsmasq` | `config/dnsmasq/` |
| VPN | wireguard-tools | `config/wireguard/` |
+731 -167
View File
File diff suppressed because it is too large Load Diff
+126 -46
View File
@@ -6,13 +6,17 @@ Vacuum Wall is a zone-based firewall appliance with a built-in SSL reverse proxy
## Architecture Overview
Vacuum Wall is built around five integrated subsystems managed through a two-layer architecture: a non-privileged Flask web UI and a privileged background daemon (`vacuum-walld`). The web UI communicates with the daemon via a Unix socket. The daemon handles all privileged operations (sudo) for the subsystems: the traffic plane uses firewalld with its nftables backend (zone-based policies, source NAT, destination NAT); the DNS/DHCP plane serves private subnets via dnsmasq; the network plane uses systemd-networkd for static IP management; the proxy plane runs nginx with automatic ACME certificates through acme.sh; and the VPN plane uses WireGuard (wg-quick) for encrypted tunnel management. All subsystems are configured and monitored through the Flask web UI, which is itself proxied through nginx with basic HTTP authentication.
Vacuum Wall is built around six integrated subsystems managed through a two-layer architecture: a non-privileged Flask web UI and a privileged background daemon (`vacuum-walld`). The web UI communicates with the daemon via a Unix socket; the daemon also streams real-time state over a local WebSocket (127.0.0.1:9091) — a full `snapshot` on connect, then per-subsystem `versions` (structural) and `tick` (volatile-only) deltas — so the UI auto-refreshes without HTTP polling. The daemon handles all privileged operations (sudo) for the subsystems: the traffic plane uses firewalld with its nftables backend (zone-based policies, source NAT, destination NAT); the DNS/DHCP plane serves private subnets via dnsmasq; the network plane uses systemd-networkd for static IP management; the proxy plane runs nginx with automatic ACME certificates through acme.sh; the VPN plane uses WireGuard (wg-quick) for encrypted tunnel management; and the authentication subsystem manages users, passkeys, and JWT sessions. Certificate management is tracked as a standalone state subsystem with its own API. In total, `lib/state.py` tracks 7 state subsystems. All subsystems are configured and monitored through the Flask web UI, which is itself proxied through nginx (TLS termination only — management authentication is a Flask-layer JWT, not nginx basic auth; individual proxy domains may optionally configure their own basic auth).
## Subsystems
### Firewall
The firewall uses firewalld's zone model for traffic control. Network interfaces are assigned to zones such as external, internal, VPN, and trusted. Rules and services define which traffic is allowed between zones. Source NAT (masquerade) enables RFC 1918 networks to reach the internet through the external interface. Destination NAT rules provide port forwarding, exposing internal services to external networks on configurable ports.
The firewall uses firewalld's zone model for traffic control. Network interfaces are assigned to zones such as external, internal, and trusted, plus a per-access-class `vpn-<class>` zone for each WireGuard access class (managed by the WireGuard sync). Zones carry an optional per-zone `target` (accept/drop/reject), and rules express fine-grained policies via services, port rules, and rich rules. Source NAT (masquerade) enables RFC 1918 networks to reach the internet through the external interface. Destination NAT rules provide port forwarding, exposing internal services to external networks on configurable ports.
**Interface-coverage invariant.** Every network-managed interface (`lo`/`wg*` excluded) must be covered by a zone in the firewall config or declared in the top-level `unmanaged` list. The invariant is enforced at save time (400) and at apply time (409; `{"force": true}` overrides); live drift is advisory only and surfaced as `uncovered_interfaces` in state.
**Pending-changes model.** Edits saved to a config are not applied until the operator applies them. Each subsystem exposes `pending_changes` plus a `pending_diff` of the changed fields, aggregated at `GET /api/status/pending`. `POST /api/status/apply-all` applies pending changes in dependency order (networkd → firewall → wireguard → dnsmasq → nginx); `POST /api/status/cancel-all` reverts all pending edits to the last-applied config.
### DHCP/DNS
@@ -20,96 +24,157 @@ dnsmasq serves as both the DHCP server and local DNS resolver. It is configured
### SSL Proxy
The nginx reverse proxy handles HTTPS termination for user-defined domains, with certificates automatically provisioned and renewed via acme.sh and an ACME provider (Let's Encrypt by default). Each proxy domain is configured with an HTTP-to-HTTPS redirect, modern TLS settings, and a configurable backend target. New proxy domains are added through the web UI, and the configuration is applied without manual intervention.
The nginx reverse proxy handles HTTPS termination for user-defined domains, with certificates automatically provisioned and renewed via acme.sh and the configured ACME provider (the CA is config-driven; the code default is Let's Encrypt). The configuration is a three-part model: named `backends`, `domains` that reference them, and a global `ssl` settings block. Each domain's paths resolve against its named backend's path table, and a builtin `webui` backend serves the management interface (Flask on 127.0.0.1:9090 plus the WebSocket on 127.0.0.1:9091). Backends are managed through the web UI (list, add, update, remove). Proxy domains may additionally gate paths with per-domain basic auth via a generated `.htpasswd` file — never on the management domain, which relies on the Flask-layer JWT. New proxy domains are added through the web UI, and the configuration is applied without manual intervention.
### Network (systemd-networkd)
The networkd subsystem manages static IP configuration for network interfaces via systemd-networkd. It renders declarative JSON configuration into per-interface `.network` INI files (`50-<name>.network`), supporting static addresses, routes, DNS, DHCP clients, link settings, and all `[Address]`, `[Route]`, `[DHCPv4]`, `[DHCPv6]`, and `[Link]` section keys. When the full apply runs, public DNS servers from networkd configs are auto-synced to dnsmasq's upstream resolvers. Helper endpoints can infer candidate DHCP ranges from static IPs and suggest firewalld zone assignments based on interface role.
The networkd subsystem manages static IP configuration for network interfaces via systemd-networkd. It renders declarative JSON configuration into per-interface `.network` INI files (`99-<name>.network`), supporting static addresses, routes, DNS, DHCP clients, link settings, and all `[Address]`, `[Route]`, `[DHCPv4]`, `[DHCPv6]`, and `[Link]` section keys. When the handler applies an interface, it removes lower-priority conflicting `.network` files from the system directory. When the full apply runs, public DNS servers from networkd configs are auto-synced to dnsmasq's upstream resolvers. Helper endpoints can infer candidate DHCP ranges from static IPs and suggest firewalld zone assignments based on interface role.
### WireGuard
WireGuard support provides server-side VPN tunnel management. Peers are added through the web UI, with the system generating client configuration files that can be downloaded and applied on remote devices. The dashboard displays active connections and transfer statistics for each peer, allowing operators to monitor tunnel health and usage.
WireGuard support provides server-side VPN tunnel management. Tunnels are organized into **access classes**: each class owns a `wg-<class>` interface, a `vpn-<class>` firewall zone, a dedicated subnet, listen port, and keypair, plus a `lan_access` flag controlling whether its peers can reach the LAN. Two classes exist by default (`full`, with LAN access, and `internet`, without). Classes are managed through CRUD endpoints (add, update, delete, reorder, generate keys). Peers are assigned to a class and added through the web UI, with the system generating client configuration files that can be downloaded and applied on remote devices. The dashboard displays active connections and transfer statistics for each peer, allowing operators to monitor tunnel health and usage.
### Authentication
Authentication is a first-class subsystem. Users, per-subsystem read/rw permissions, Argon2id password hashes, and optional passkeys (WebAuthn/FIDO2) are stored in a SQLite database (`data/auth.db`), reached through an abstract database layer that never exposes raw SQL. Sessions use JWT access + refresh tokens: each user holds their own HS256 signing secret, and revoked tokens are blacklisted by `jti`. A builtin `admin` user is seeded at bootstrap. The subsystem exposes `/api/auth/*` endpoints and the login, users, and passkeys pages.
### Certificates (ACME)
Certificate management is a standalone state subsystem with its own API (`/api/certs/*`). acme.sh issues and renews certificates for proxy domains against the configured CA provider; self-signed certificates can be generated for domains without an ACME account, and ACME accounts can be registered or deactivated. A systemd timer runs periodic renewals, and certificate state (issuance, expiry) is collected like any other subsystem.
## Tech Stack
- Debian 13 (trixie) target platform
- Python 3.13+, Flask 3.x for web management
- aiohttp (daemon server) + requests-unixsocket (Unix-socket client)
- firewalld (nftables backend)
- systemd-networkd (ip-lladdr, networkctl)
- nginx 1.26+
- systemd-networkd (networkctl)
- nginx
- dnsmasq
- WireGuard tools (wireguard-tools)
- acme.sh for ACME certificate management (Let's Encrypt by default)
- acme.sh for ACME certificate management (CA provider config-driven; code default Let's Encrypt)
- SQLite (auth database)
- PyJWT (JWT sessions), argon2-cffi (Argon2id password hashing), webauthn (passkeys), passlib (htpasswd only)
- htm.js (vendored JS tagged-template HTML adapter)
## Quick Start
To install Vacuum Wall on a Debian 13 system, run `install.sh` as root with required settings (CLI flags or environment variables):
To install Vacuum Wall on a Debian 13 system, run `scripts/install.sh` as root with required settings (CLI flags or environment variables):
```bash
# Production
./install.sh --mgmt-pass yourpassword
./scripts/install.sh --mgmt-pass yourpassword
# Development (auto-detects your user)
./install.sh --dev --mgmt-pass yourpassword
./scripts/install.sh --dev --mgmt-pass yourpassword
```
After installation, access the management interface at `https://<hostname>.local` using the credentials you configured. The `install.sh` script auto-detects the system hostname, network interfaces, and provisions nginx, authentication, an initial self-signed certificate, and all services. Run `./install.sh --help` for all options.
After installation, access the management interface at `https://<hostname>.local` using the credentials you configured. The `scripts/install.sh` script auto-detects the system hostname, network interfaces, and provisions nginx, authentication, an initial self-signed certificate, and all services. Run `./scripts/install.sh --help` for all options.
## Project Structure
```
├── install.sh # Deployment script (renders Jinja2 templates)
├── README.md # Project overview
├── AGENTS.md # Agent instructions
├── .gitignore
├── pyproject.toml # Project metadata + dependencies
├── .venv/ # Python virtual environment
├── scripts/ # Utility scripts
│ ├── install.sh # Deployment script (renders Jinja2 templates)
│ ├── update-vendor.sh # Download vendored libraries (acme.sh, htm)
│ ├── bootstrap_auth.py # Auth DB bootstrap (creates the operator user)
│ └── restart-services.sh # Restart installed system services
├── config/ # Declarative JSON configuration (source of truth)
│ ├── firewall/ # Firewall zone & rule config
│ ├── dnsmasq/ # DHCP/DNS config
│ ├── network/ # systemd-networkd per-interface config
│ ├── firewall/ # Firewall zone & rule config
│ ├── nginx/ # Proxy domain & SSL config
── wireguard/ # VPN interface & peer config
│ ├── nginx/ # Proxy backend, domain & SSL config
│ ├── wireguard/ # VPN access-class, interface & peer config
── acme/ # ACME account settings (email, CA provider)
│ └── auth/ # Authentication settings (JWT, WebAuthn)
├── data/ # Runtime artifacts & generated files
│ ├── auth.db # SQLite auth database (users, passkeys)
│ ├── certs/ # Management-domain TLS keypair
│ ├── daemon.sock # Daemon Unix socket
│ ├── nginx/sites-enabled/ # Generated server blocks
│ ├── nginx/.htpasswd # Basic-auth entries for proxy domains
│ ├── dnsmasq/fragments/ # User config fragments
│ ├── acme/ # ACME certificates
│ ├── firewall/ # Firewall rule backup
│ ├── logs/ # Application logs
│ ├── networkd/ # Generated 50-<name>.network files
│ └── wireguard/ # Generated WireGuard configs
│ ├── acme/ # acme.sh home: certs, account, webroot (www/)
│ ├── firewall/rules.json # Pre-apply recovery snapshot
│ ├── networkd/ # Generated 99-<name>.network files
│ ├── wireguard/ # Generated WireGuard configs
│ └── logs/ # Application logs
├── daemon/ # Privileged background daemon
│ ├── server.py # aiohttp server, cache, batch routing, handler registry
│ ├── server.py # aiohttp server: endpoint registry (daemon/iface.py), batch routing, WebSocket broadcast (snapshot/versions/tick), state refresh, per-subsystem polling
│ ├── client.py # Sync HTTP client over Unix socket
│ ├── iface.py # Single source of truth for daemon API endpoints
│ ├── __main__.py # Module entry point (python -m daemon.server)
│ ├── handlers/ # Privileged operation handlers (all sudo calls)
│ │ ── network.py # networkd handler (generate + apply)
├── system/ # System file templates (all Jinja2)
│ │ ── firewall.py # Zone/rich-rule CRUD + apply
│ │ ├── dnsmasq.py # DHCP/DNS config + apply
│ │ ├── nginx.py # Proxy domain/backend + SSL apply
│ │ ├── network.py # networkd handler (generate + apply)
│ │ ├── wireguard.py # Access-class/peer CRUD + tunnel control
│ │ ├── acme.py # Certificate issue/renew/self-signed, account
│ │ ├── auth.py # User/passkey management
│ │ ├── logs.py # Log streaming
│ │ ├── status.py # Pending/apply-all/cancel-all
│ │ ├── system.py # System info & metrics
│ │ └── common.py # Shared handler helpers (sync emit + refresh)
│ └── collectors/ # Read-only per-subsystem state collectors
│ ├── firewall.py # firewall collector
│ ├── dnsmasq.py # dnsmasq collector
│ ├── networkd.py # networkd collector
│ ├── nginx.py # nginx collector
│ ├── wireguard.py # wireguard collector
│ ├── acme.py # acme collector
│ └── system.py # system collector
├── system/ # System file templates (mostly Jinja2)
│ ├── systemd/ # Service and timer unit files
│ │ ├── vacuum-wall.service # Web UI service (rendered at install)
│ │ ├── vacuum-wall-acme.service # Certificate renewal (rendered at install)
│ │ ── vacuum-wall-acme.timer # Renewal schedule
│ │ ── vacuum-wall-acme.timer # Renewal schedule
│ │ └── vacuum-walld.service # Privileged daemon (rendered at install)
│ ├── sudoers.d/ # Sudo whitelist (rendered at install)
│ ├── tmpfiles.d/ # tmpfiles.d spec (installed verbatim, not Jinja)
│ ├── nginx/ # Nginx config templates (rendered at runtime)
│ ├── dnsmasq.conf # Dnsmasq template (rendered at runtime)
── wireguard*.conf # WireGuard templates (rendered at runtime)
── wireguard.conf # WireGuard server template (rendered at runtime)
│ ├── wireguard-client.conf# WireGuard client template (rendered at runtime)
│ ├── acme-deploy.py # ACME deploy hook (installed verbatim, not Jinja)
│ └── acme-deploy.sh # ACME deploy wrapper (installed verbatim, not Jinja)
├── lib/ # Subsystem abstraction layer
│ ├── common.py # Shared utilities (run, run_proc, load_json, save_json, deep_merge, ensure_dirs)
│ ├── common.py # Shared utilities (run, run_proc, load_json, save_json, deep_merge, ensure_dirs, get_interface_ip, config_hash, stamp_applied, strip_apply_meta, compute_pending, deep_diff, revert_to_applied, validate_interface_name)
│ ├── logging.py # Logging setup
│ ├── firewall.py # firewalld bindings
│ ├── network.py # systemd-networkd rendering & parsing
│ ├── dnsmasq.py # DHCP/DNS configuration
│ ├── nginx.py # Reverse proxy configuration
│ ├── state.py # State collector (uses lib.network.parse_networkctl_status)
│ ├── acme.py # Certificate management
── wireguard.py # VPN tunnel management
│ ├── nginx.py # Reverse proxy configuration (backends model)
│ ├── state.py # In-memory state store (per-subsystem data, version counters, two-layer versions/tick diff, poll intervals, volatile registration); collectors live in daemon/collectors/
│ ├── sync.py # Cross-subsystem event bus
── acme.py # Certificate management (ACME helpers)
│ ├── wireguard.py # VPN tunnel and peer management
│ ├── system_import.py # Startup reconciler (imports live system configs into JSON)
│ ├── bootstrap.py # Daemon-startup filesystem bootstrap
│ ├── schema.py # TypedDict state schemas
│ ├── auth.py # JWT access+refresh tokens, per-user HS256 secrets, jti blacklist
│ ├── auth_users.py # Multi-user management, per-subsystem read/rw permissions, builtin admin
│ ├── password.py # Argon2id password hashing
│ ├── webauthn.py # Passkey (FIDO2/WebAuthn) support
│ ├── db.py # Abstract database layer (opaque query IDs)
│ └── db_sqlite.py # SQLite backend (data/auth.db)
├── webui/ # Flask web application
│ ├── server.py # Application entry point
│ ├── api/ # REST API route modules (blueprints)
│ │ ├── common.py # Shared API response helpers (_ok, _error)
│ │ ├── firewall.py # Firewall API
│ │ ├── dhcp.py # DHCP/DNS API
│ │ ├── proxy.py # Nginx proxy API
│ │ ├── proxy.py # Nginx proxy API (domains + backends)
│ │ ├── certs.py # Certificate API
│ │ ├── wireguard.py # WireGuard API
│ │ ├── network.py # Networkd API
│ │ ── logs.py # Logs API
│ │ ── logs.py # Logs API
│ │ ├── auth.py # Authentication API
│ │ └── status.py # Status API (pending/apply-all/cancel-all)
│ └── static/ # SPA (index.html, app.js, style.css)
│ ├── hoover/ # Hoover SPA framework (VDOM, reactivity, router, components)
│ │ ├── index.js # Barrel export of all public APIs
@@ -121,18 +186,32 @@ After installation, access the management interface at `https://<hostname>.local
│ │ ├── websocket.js
│ │ ├── api.js
│ │ ├── helpers.js
│ │ ── components/ # Layout, data display, modal, toast
└── pages/ # Page modules (each defines a route via definePage)
├── docs/ # Documentation
├── overview.md # This file
├── deployment.md
├── api.md
├── security.md
├── architecture.md
├── config.md
└── hoover.md # Hoover SPA framework
── scripts/ # Utility scripts
── update-vendor.sh # Vendor frontend library updates
│ │ ── html.js # htm.js tag adapter
│ ├── model.js # Reactive model store
│ │ ├── auth_model.js# Auth session model
│ ├── dirty.js # Dirty-state tracking
│ ├── schema.js # Schema validation helpers
│ └── components/ # applyconfirm, auth, data, layout, modal, qr, toast
└── pages/ # 15 page modules (each defines a route via definePage):
# dashboard, zones, rules, nat, interfaces, dhcp,
# proxy, backends, certs, wireguard, logs, login,
# users, passkeys, notfound
── vendor/ # Vendored scripts and JS libraries
── acme.sh # ACME certificate client
│ ├── htm.js # JS tagged-template HTML adapter
│ └── qrcode-svg-1.1.0.js # QR code generation (SVG)
├── tests/ # Test suites
│ ├── test_*.py # 28 Python modules (pytest; subprocess calls mocked)
│ └── test-*.js # 9 JS test modules (hoover framework)
└── docs/ # Documentation
├── overview.md # This file
├── deployment.md
├── api.md
├── security.md
├── architecture.md
├── config.md
├── state-model.md # State schema, versions/tick diff, pending-changes model
└── hoover.md # Hoover SPA framework
```
## Documentation
@@ -142,4 +221,5 @@ After installation, access the management interface at `https://<hostname>.local
- [Security Model](security.md) - Privilege model and sudo whitelist
- [Architecture](architecture.md) - Detailed subsystem design
- [Configuration](config.md) - Config file formats and locations
- [State Model](state-model.md) - State schema, versions/tick diff, pending-changes
- [Hoover Framework](hoover.md) - Frontend SPA framework reference
+114 -32
View File
@@ -7,10 +7,12 @@ Vacuum Wall uses two distinct system users bridged by a shared group (the WebUI
- **`vacuum-walld`** (daemon user): Runs the `vacuum-walld` background daemon, which is the only process with sudo access. The daemon communicates with the WebUI over a Unix socket at `data/daemon.sock`. All privileged operations — firewall rule changes, nginx reloads, dnsmasq config writes, WireGuard tunnel management — are executed by the daemon through a restricted sudo whitelist at `/etc/sudoers.d/vacuum-walld`.
- **WebUI user** (default: repo owner in `--dev` mode): Runs the Flask management WebUI. Has **zero** sudo access. If the WebUI process is compromised, an attacker cannot invoke sudo directly — they are confined to the sandboxed Flask process with no privilege escalation path.
ACME certificate operations via `acme.sh` run as the WebUI user — not as root, and not as the daemon user. The automated renewal timer (`vacuum-wall-acme.timer`) runs `acme.sh --cron` as `{{ USER_NAME }}`. When triggered from the WebUI or daemon, acme.sh also runs as the non-root process invoking it, using webroot validation that does not require binding to privileged ports.
ACME certificate operations via `acme.sh` run as the daemon user (`{{ USER_DAEMON_NAME }}`) — never as root, and never from the WebUI process (the WebUI never invokes acme.sh directly). The automated renewal timer (`vacuum-wall-acme.timer`) runs `acme.sh --cron` as `{{ USER_DAEMON_NAME }}`. Issuance and renewal triggered from the WebUI are executed by the daemon as its own subprocess, using webroot validation that does not require binding to privileged ports; the only sudo call around acme.sh is the `chmod g+rwX` that reopens group access on the ACME home (see Sudo Whitelist).
This design follows the principle of least privilege: only the daemon process holds sudo access, and only for explicitly enumerated commands. The WebUI user is completely isolated from sudo.
Authentication (JWT validation, token blacklist check, permission verification) is performed at the Flask layer — not the daemon. The daemon only receives requests from the Flask process over the Unix socket, which carries **no authentication of its own**: access to it is protected purely by the socket's `0660` mode and shared-group ownership. The JWT handshake exists on the daemon's **WebSocket** endpoint: WebSocket connections to the daemon require a JWT access token, sent as the raw `Sec-WebSocket-Protocol` subprotocol name (the legacy `Bearer <token>` subprotocol and an `X-Auth-Token` header fallback are also accepted), validated before the socket upgrades.
## Communication Between WebUI and Daemon
The WebUI communicates with the daemon via synchronous HTTP requests over a Unix socket (`data/daemon.sock`), owned by `vacuum-walld:<group>` with mode `0660`. The shared group membership allows the WebUI user to connect to the socket. The daemon runs an `aiohttp` server that routes requests to handler modules (`daemon/handlers/*.py`), which execute the privileged commands.
@@ -24,34 +26,40 @@ The file `/etc/sudoers.d/vacuum-walld` grants the daemon user (`vacuum-walld`) p
| Firewall | `firewall-cmd *` | All firewalld operations (zone management, rules, services, ports) |
| Nginx | `nginx -s reload` | Graceful nginx configuration reload |
| Nginx | `nginx -t` | Nginx configuration syntax validation |
| Nginx file ops | `cp -- * /etc/nginx/`, `/etc/nginx/conf.d/`, `/etc/nginx/snippets/` | Copy rendered config files to system paths |
| Nginx file ops | `rm /etc/nginx/conf.d/vacuum-wall.conf`, `/etc/nginx/snippets/vacuum-wall-ssl.conf` | Clean up generated nginx config files |
| Nginx file ops | `chown root:root /etc/nginx/snippets/vacuum-wall-ssl.conf` | Ensure correct ownership of SSL snippet |
| Dnsmasq | `systemctl reload dnsmasq` | Apply updated dnsmasq configuration |
| Dnsmasq | `systemctl is-active dnsmasq` | Check dnsmasq service status |
| Dnsmasq file ops | `cp -- * /etc/dnsmasq.d/` | Copy rendered config files |
| Dnsmasq file ops | `tee /etc/dnsmasq.d/vacuum-wall.conf` | Write dnsmasq configuration |
| Nginx status | `systemctl is-active nginx` | Check nginx service status |
| Nginx file ops | `cp -- /run/vacuum-wall/include.tmp /etc/nginx/conf.d/vacuum-wall.conf` | Copy the rendered config include to its system path (pinned source and destination) |
| Nginx file ops | `cp -- /run/vacuum-wall/ssl-snippet.tmp /etc/nginx/snippets/vacuum-wall-ssl.conf` | Copy the rendered SSL snippet to its system path (pinned source and destination) |
| Nginx file ops | `rm /etc/nginx/conf.d/vacuum-wall.conf`, `rm /etc/nginx/snippets/vacuum-wall-ssl.conf` | Clean up generated nginx config files |
| Nginx file ops | `chown root:root /etc/nginx/conf.d/vacuum-wall.conf`, `chown root:root /etc/nginx/snippets/vacuum-wall-ssl.conf` | Ensure correct ownership of nginx config files |
| Dnsmasq | `systemctl restart dnsmasq` | Apply updated dnsmasq configuration |
| Dnsmasq status | `systemctl is-active dnsmasq` | Check dnsmasq service status |
| Dnsmasq file ops | `mkdir -p /etc/dnsmasq.d` | Ensure target directory exists |
| Dnsmasq leases | `cat /var/lib/dnsmasq/dnsmasq.leases` | Read dnsmasq lease table |
| Dnsmasq file ops | `cp -- /run/vacuum-wall/dnsmasq.tmp /etc/dnsmasq.d/vacuum-wall.conf` | Copy the rendered dnsmasq fragment to its system path (pinned source and destination) |
| Dnsmasq leases | `cat /var/lib/misc/dnsmasq.leases` | Read dnsmasq lease table |
| WireGuard | `wg-quick *` | WireGuard tunnel lifecycle (up, down, save, show) |
| WireGuard | `wg *` | WireGuard status and peer management |
| WireGuard file ops | `cp -- * /etc/wireguard/` | Copy rendered config files |
| WireGuard file ops | `cp -- /run/vacuum-wall/wg0.conf.tmp /etc/wireguard/wg0.conf` | Copy the rendered WG config to its system path (pinned source and destination) |
| WireGuard file ops | `chown root:root /etc/wireguard/wg0.conf` | Ensure correct ownership of WG config |
| Certificates | (none) | acme.sh runs as the non-root service user directly; no sudo escalation is needed (webroot validation is used) |
| Certificates | `chmod g+rwX {{ ACME_HOME }}/*` | Reopen group read/write on ACME home files after acme.sh hardens them to owner-only modes (`normalize_acme_home()`, run before every daemon acme.sh invocation). Files only: setgid directories already grant group rwx |
| Network queries | `ip -o link show` | List network interfaces |
| Network queries | `ip -o addr show` | List IP addresses on interfaces |
| Network queries | `ip -o addr show *` | Query IP address for a specific interface (DHCP gateway auto-population) |
| Networkd | `networkctl status *` | Query interface status from networkd |
| Networkd | `networkctl reload *` | Reload networkd for a specific interface |
| Networkd | `networkctl reload` | Reload networkd for all interfaces |
| Networkd | `networkctl reconfigure *` | Reconfigure a specific interface |
| Networkd file ops | `cp -- /run/vacuum-wall/99-*.network /etc/systemd/network/` | Copy rendered network unit files (pinned destination dir, `99-*` source pattern) |
| Networkd file ops | `rm /etc/systemd/network/*.network` | Remove stale network unit files |
| Networkd file ops | `mkdir -p /etc/systemd/network` | Ensure target directory exists |
| Sysctl | `sysctl -w *` | Set kernel parameters |
| Logs | `journalctl --unit=* -n *` | Query systemd journal for managed services |
| Logs | `cat /var/log/nginx/*` | Read nginx access and error logs |
Key safety properties:
- Each `Cmnd` entry specifies the full path to the binary (e.g., `/usr/bin/firewall-cmd`).
- Wildcard entries exist only for commands where the full argument space is needed (`firewall-cmd *`, `wg-quick *`, `wg *`), but none grant shell access or arbitrary command execution.
- Full-argument wildcard entries exist only for commands where the full argument space is needed (`firewall-cmd *`, `wg-quick *`, `wg *`, `sysctl -w *`, `journalctl --unit=* -n *`, `networkctl status *`, `networkctl reconfigure *`, `ip -o addr show *`); the remaining wildcard entries target fixed destination paths with a filename pattern (`cp -- /run/vacuum-wall/99-*.network /etc/systemd/network/`, `rm /etc/systemd/network/*.network`, `chmod g+rwX {{ ACME_HOME }}/*`). All file-copy entries are pinned to a single source file under the daemon-owned `/run/vacuum-wall` runtime dir and a single destination path. None of the entries grant shell access or arbitrary command execution.
- `NOPASSWD` is used so the application never prompts for a password. `Defaults:<user>` restricts the secure path and disables TTY requirement.
- The sudoers file is rendered from a Jinja2 template at install time, substituting the configured user name.
- The sudoers file is rendered from a Jinja2 template at install time, substituting the configured `USER_DAEMON_NAME` and `ACME_HOME` variables (the install also renders `USER_NAME`, `USER_GROUP`, and `PROJECT_DIR` for the systemd unit templates).
## Daemon Client Path Resolution
@@ -61,23 +69,70 @@ The `daemon/client.py` module resolves `<param>` placeholders in URL paths befor
### Management Interface
The Flask WebUI binds exclusively to `127.0.0.1:9090`. It is not exposed directly to any network interface. All external access to the management UI is routed through an nginx reverse proxy on the designated management domain, which provides SSL termination and HTTP Basic Authentication. The `.htpasswd` file is stored at `data/nginx/.htpasswd`.
The Flask WebUI binds exclusively to `127.0.0.1:9090`. It is not exposed directly to any network interface. All external access to the management UI is routed through an nginx reverse proxy on the designated management domain, which provides SSL termination. Authentication is handled at the Flask layer via JWT validation — no nginx-level `auth_basic` is applied to the management domain. Static assets under `/static/` are served directly by nginx from `webui/static/` (unauthenticated, the same exposure as the Flask static route) with `Cache-Control: no-cache`, `X-Content-Type-Options: nosniff`, and a restrictive `Content-Security-Policy: default-src 'none'`.
The management interface does not set security hardening headers (e.g., `X-Content-Type-Options`, `X-Frame-Options`, HSTS). It relies on nginx basic authentication, SSL termination, and the systemd sandbox for its security boundary.
JWT tokens are stored in browser `sessionStorage` and injected as `Authorization: Bearer <token>` headers. The API **never** reads cookies — authentication is header-only. This eliminates CSRF concerns: cross-origin requests cannot set custom headers.
Flask sets a full `Content-Security-Policy` (all sources locked to `'self'` with `img-src 'self' data:`) and `X-Content-Type-Options: nosniff` on **every** response via an `after_request` hook — the CSP includes `frame-ancestors 'none'`, `base-uri 'self'`, and `form-action 'self'`. `X-Frame-Options` and HSTS are absent on the management domain; clickjacking protection comes from the CSP `frame-ancestors 'none'` directive instead. The SPA relies on JWT authentication, SSL termination, and the systemd sandbox for its security boundary.
The auth-exempt public path list covers the SPA root, static and vendor files, `POST /api/auth/login`, `POST /api/auth/refresh`, and the two WebAuthn authentication endpoints (`POST /api/auth/webauthn/authenticate-begin`, `POST /api/auth/webauthn/authenticate-finish`). nginx writes the management domain's traffic to dedicated `wall_mgmt_access.log` / `wall_mgmt_error.log` files; non-management domains get per-domain `<domain>_access.log` / `<domain>_error.log` logs.
### Proxy Domains
Every proxied domain configured in Vacuum Wall enforces:
Proxied domains **without** a management path enforce, at the nginx server level:
- **HTTP-to-HTTPS redirect** — All HTTP requests return a 301 Permanent Redirect to the HTTPS equivalent.
- **HTTP Strict Transport Security (HSTS)** — The `Strict-Transport-Security` header is set with a long max-age and `includeSubDomains` to prevent downgrade attacks.
- **Security headers** on all proxied responses:
- **HTTP-to-HTTPS redirect** rendered only when the domain has `force_ssl` enabled. All HTTP requests return a 301 Permanent Redirect to the HTTPS equivalent (the HTTP server block also serves the ACME HTTP-01 challenge location `/.well-known/acme-challenge/` before the redirect).
- **HTTP Strict Transport Security (HSTS)** — The `Strict-Transport-Security` header is set with `max-age=31536000; includeSubDomains` to prevent downgrade attacks.
- **Security headers** on all responses from the domain:
- `X-Content-Type-Options: nosniff` — Prevents MIME-type sniffing.
- `X-Frame-Options: DENY` — Prevents clickjacking via iframes.
- `X-XSS-Protection: 1; mode=block` — Enables browser XSS filtering.
- `Referrer-Policy: strict-origin-when-cross-origin` — Limits referrer information leakage.
Additional proxy headers (`extra_headers` in the domain config) are delivered to the upstream backend via nginx `proxy_set_header` directives — they are not sent as response headers to clients.
Domains that carry a management path get none of the above — the management SPA receives its security headers from Flask instead (see Management Interface).
**Basic auth on proxy domains**: a domain-level `auth` block renders `auth_basic` + `auth_basic_user_file` on the whole server block, and per-path `auth` blocks apply it to individual proxied paths. The generated `.htpasswd` files hash passwords with **SHA-256 crypt** (mode 0640). The management domain never gets `auth_basic` — management auth is the Flask-layer JWT middleware.
Additional proxy headers (`headers` in the path-level config) are delivered to the upstream backend via nginx `proxy_set_header` directives — they are not sent as response headers to clients.
### JWT Authentication Lifecycle
JWT-based authentication replaces HTTP Basic Auth for the management WebUI. The token lifecycle is:
1. **Login**: User submits credentials via `POST /api/auth/login`. The daemon verifies the password hash (Argon2id) against `data/auth.db`. On success, an access token (5 min — the fresh-install bootstrap writes `access_token_ttl: 300`; TTLs are configurable in `config/auth/config.json`) and a refresh token (7 days) are issued, each bound to a fresh `session_id`.
2. **Validation**: Every API request to Flask includes `Authorization: Bearer <token>` and an `X-Session-Id` header. The `before_request` middleware returns 401 without the session header, validates the token signature, checks expiry, verifies the `X-Session-Id` matches the token's `session_id` claim (binding the token to the browser session that created it), queries the SQLite `token_blacklist` table, and verifies per-subsystem permissions.
3. **Auto-refresh**: Before the access token expires, the frontend's `scheduleRefresh()` timer (fires at TTL 60s, minimum 30s) calls `POST /api/auth/refresh` with the refresh token and `session_id` — the refresh endpoint requires a matching `session_id` so a stolen refresh token cannot be rotated without the originating session. The old refresh token is blacklisted and a new pair is issued. At page load/restore, if the stored access token is rejected (401) on the session check, the frontend performs exactly one refresh from the stored refresh token before falling to the login page.
4. **Revocation**: The primary revocation mechanism is **per-user JWT signing-secret rotation**: tokens are signed with a per-user secret (not a global key), and changing the password or resetting it, or changing permissions, rotates the user's secret (deleting the user removes the secret entirely), immediately invalidating every existing access and refresh token. The affected user's active refresh token `jti` is additionally inserted into `token_blacklist`, as is the access token's `jti` on logout (`POST /api/auth/logout`). On refresh rotation the old refresh token's `jti` is blacklisted and the new token replaces the stored row in `refresh_tokens`. One row per user means each user has a single active refresh session: a refresh from a second tab overwrites the first tab's row, and logout blacklists whichever token is currently stored. Expired blacklist entries are cleaned by the daemon's polling loop (every 60s) and by a probabilistic check inside `blacklist_token()`.
Token theft protection:
- Short-lived access tokens (5 min) limit the window of exploitation
- Per-user signing-secret rotation on password/permission change plus the token blacklist prevent reuse after credential changes or logout
- `X-Session-Id` binding ties access and refresh tokens to the originating browser session
- XSS mitigations: CSP headers set by Flask on every response
**WebSocket session binding limitation**: WebSocket connections skip `session_id` validation. Browsers cannot send custom headers during the WebSocket handshake — the bundled client passes the raw JWT as the `Sec-WebSocket-Protocol` subprotocol name (a JWT is a valid RFC 6455 token; the `Bearer ` prefix is not, so it cannot be used) (a custom nginx setup may instead inject it as `X-Auth-Token`). This means a stolen access token can be used to open WebSocket connections for the full 5-minute TTL without session verification. Short-lived TTL and management domain CSP headers mitigate this risk.
### WebAuthn Security
WebAuthn (passkeys) provides passwordless authentication via the browser's Web Authentication API. Security properties:
- **Credential binding**: Each credential is cryptographically bound to the specific `rp_id` (management domain) and `origin` (HTTPS URL). Credentials cannot be phished to a different domain.
- **Private key protection**: The private key never leaves the authenticator device. The server stores the `username`, `credential_id`, display `name`, `transports`, public key, and signature counter in the `webauthn_creds` table.
- **Assertion verification**: Each authentication attempt verifies the signature against the stored public key and checks that the signature count has increased (replay prevention).
- **RP configuration**: `rp_id` and `origin` are **derived from the request** (`X-Forwarded-Proto`/`X-Forwarded-Host`) and validated against the live management domains, so credentials are bound to the domain the user actually reached. The `webauthn` section of `config/auth/config.json` holds only `enabled` and `rp_name` (the installer writes `rp_id`/`origin` on fresh install, but the runtime never reads them).
- **Fallback**: Password authentication always remains available as a fallback. Losing a WebAuthn credential does not lock the user out.
### Header-Only Authentication and CSRF
The API exclusively reads the `Authorization` header — never cookies. This architecture eliminates CSRF risk:
- Cross-site requests cannot set custom HTTP headers due to browser CORS restrictions
- No cookie-based session to exploit
- No SameSite, double-submit, or origin checking needed
**XSS as the primary attack surface**: With header-only auth, XSS is the primary attack vector since `sessionStorage` is accessible to page scripts. Mitigations include:
- CSP headers set by Flask's `after_request` hook on every API/SPA response (nginx adds a separate `default-src 'none'` CSP only on `/static/`)
- Short-lived access tokens (5 min) with secret rotation and blacklist on logout
### TLS Configuration
@@ -88,6 +143,15 @@ The default nginx SSL configuration enforces modern TLS only:
- **ssl_prefer_server_ciphers** defaults to `off` (client chooses).
- **Session settings**: `ssl_session_timeout 1d`, `ssl_session_cache shared:TLS:10m`, `ssl_session_tickets off`.
### Brute-Force Protection
Login and WebAuthn authentication attempts are rate-limited in-process with sliding windows that count failures only (a success resets the bucket):
- **Password login**: 10 failures per 300s, tracked per **username and per client IP** (`X-Real-IP`).
- **WebAuthn**: 5 failures per 600s, tracked per username and per client IP.
To prevent username enumeration, password verification for a nonexistent user runs a dummy Argon2id verification against a pre-computed hash, keeping timing uniform. The limiters are in-memory; counts reset on daemon restart (SIGHUP reload, process restart).
## Systemd Hardening
Both `vacuum-wall.service` (WebUI) and `vacuum-walld.service` (daemon) apply comprehensive systemd sandboxing directives to isolate their processes from the rest of the system:
@@ -95,7 +159,12 @@ Both `vacuum-wall.service` (WebUI) and `vacuum-walld.service` (daemon) apply com
| Directive | Value | Effect |
|---|---|---|
| `ProtectSystem` | `strict` | Mounts the entire file system as read-only, except explicitly allowed paths |
| `ReadWritePaths` | project dir, `/tmp`, and (WebUI only) `config/`, `data/` subdirs | The project directory and runtime paths are writable |
| `ReadWritePaths` | project dir, `/tmp`, the generated `/etc` config dirs, and the volatile `/run` entries (`/run/vacuum-wall`, `/run/firewalld`, `/run/nginx`, `/run/nginx.pid`), plus `/var/log/nginx` and `/var/log/vacuum-wall` (daemon); (WebUI only) `config/`, `data/` subdirs and `/var/log/vacuum-wall` | The project directory and runtime paths are writable. Every entry must **exist** when the unit spawns or namespace setup fails (`226/NAMESPACE`), so volatile `/run` entries are pre-created by systemd (see below). Only paths the unit genuinely writes are listed — e.g. `/run/sudo` was historically listed but is now omitted because the NOPASSWD sudo children never need it |
| `RuntimeDirectory` | `vacuum-wall nginx` (daemon only) | Creates `/run/vacuum-wall` and `/run/nginx` owned by the daemon user before namespace setup; removed on stop |
| `RuntimeDirectoryMode` | `0750` (daemon only) | Group-readable runtime dirs (the shared group owns them) |
| `LogsDirectory` | `vacuum-wall` (both units) | Creates `/var/log/vacuum-wall` owned by the service user before namespace setup |
| `ExecReload` | `/bin/kill -HUP $MAINPID` (WebUI only) | SIGHUP triggers the WebUI's auto-reload (reloads `webui.*`/`lib.*` modules, then restarts via SIGTERM); the daemon unit has no `ExecReload` |
| tmpfiles.d spec | `system/tmpfiles.d/vacuum-wall.conf` (installed to `/etc/tmpfiles.d/`, applied at early boot by `systemd-tmpfiles-setup.service`) | Pre-creates the root-owned `/run/firewalld` (`0750`) and `/run/nginx.pid` (`0644`) at early boot so the daemon's `ReadWritePaths=` entries resolve on a fresh boot (in practice firewalld, which starts first, creates the directory itself; nginx rewrites the pid file on start) |
| `PrivateTmp` | `yes` | Provides a private `/tmp` and `/var/tmp` namespace |
| `NoNewPrivileges` | `yes` | Prevents the process from gaining new privileges via `setuid`/`setgid` |
| `PrivateDevices` | `yes` | Hides all device files under `/dev` |
@@ -109,35 +178,48 @@ Both `vacuum-wall.service` (WebUI) and `vacuum-walld.service` (daemon) apply com
| `MemoryDenyWriteExecute` | `yes` | Prevents creating memory regions that are both writable and executable |
| `SystemCallFilter` | `@system-service` | Allows only a curated set of system calls safe for services |
| `RestrictRealtime` | `yes` | Prevents the process from acquiring realtime scheduling priorities |
| `RestrictAddressFamilies` | `AF_UNIX AF_INET AF_INET6` | Restricts available address families |
| `IPAddressDeny` | `any` | Drops all network traffic by default |
| `IPAddressAllow` | `localhost` | Allows only loopback communication (required to reach the other process at 127.0.0.1) |
| `RestrictAddressFamilies` | `AF_UNIX AF_INET AF_INET6` (WebUI); `AF_UNIX AF_INET AF_INET6 AF_NETLINK` (daemon) | Restricts available address families; the daemon's extra `AF_NETLINK` is its only additional network primitive |
| `IPAddressDeny` | `any` (both units) | Drops all IP traffic by default |
| `IPAddressAllow` | `localhost` (both units) | Allows only loopback communication (required to reach the other process at 127.0.0.1) |
The WebUI unit additionally restricts address families and denies all IP traffic except to localhost — it cannot reach any external network interface. Both units use template variables (`{{ USER_NAME }}`, `{{ USER_GROUP }}`, `{{ USER_DAEMON_NAME }}`, `{{ PROJECT_DIR }}`) rendered at install time.
Both units deny all IP traffic except to localhost, so neither can reach any external network interface; the only difference in network access is the daemon's extra `AF_NETLINK` family (needed for its netlink queries). Both units use template variables (`{{ USER_NAME }}`, `{{ USER_GROUP }}`, `{{ USER_DAEMON_NAME }}`, `{{ PROJECT_DIR }}`) rendered at install time.
This hardening ensures that even if either process is compromised, the attacker is confined to a sandboxed environment with no direct network access, no write access outside the project directory, and no ability to escalate privileges through kernel interfaces.
This hardening ensures that even if either process is compromised, the attacker is confined to a sandboxed environment with no direct network access, no ability to escalate privileges through kernel interfaces, and a strictly bounded write scope: outside the project directory the daemon's unit lists only `/etc/systemd/network`, `/etc/nginx`, `/etc/dnsmasq.d`, `/etc/wireguard`, `/var/log/nginx`, and `/var/log/vacuum-wall` (plus `/tmp` and the `/run` runtime entries), and the WebUI's unit lists only its `config/` and `data/` subdirs and `/var/log/vacuum-wall`.
## Network Security
### Default Deny
The firewalld default zone policy is set to deny all incoming traffic. Only explicitly allowed services and ports are accessible. Outbound traffic is permitted by default.
Incoming traffic is denied by default — this is firewalld's built-in behavior for the default zone (no Vacuum Wall code sets a zone target; `apply` only reconciles targets explicitly present in the config). Only explicitly allowed services and ports are accessible. Outbound traffic is permitted by default.
### Zone-Based Traffic Isolation
The `lib/firewall` module is a generic firewalld parser with no hardcoded zone definitions. Zone structure is defined declaratively in `config/firewall/config.json` at runtime. A typical deployment uses:
The `lib/firewall` module is a generic firewalld parser; zone structure is defined declaratively in `config/firewall/config.json` at runtime. The only hardcoded zone knowledge is `FIREWALLD_BUILTIN_ZONES` — the 9 zone names firewalld ships by default (`block`, `dmz`, `drop`, `external`, `home`, `host`, `internal`, `public`, `trusted`) — used so built-in zones are never flagged as unmanaged (not in config). The `public` zone is additionally special-cased: its masquerade state is not reconciled by `apply` and cannot be enabled through the masquerade endpoint (see IP Forwarding and NAT). A typical deployment uses:
| Zone | Interface | Purpose | Behavior |
|---|---|---|---|
| `external` | WAN (e.g., `eth0`) | Untrusted Internet-facing | Only essential services (HTTPS, WireGuard) are open. ICMP echo is rate-limited. |
| `external` | WAN (e.g., `eth0`) | Untrusted Internet-facing | Only essential services (HTTPS, WireGuard) are open. ICMP echo rate-limiting is typical in this deployment but is not enforced by any Vacuum Wall code. |
| `internal` | LAN (e.g., `eth1`) | Trusted local network | DHCP and DNS served to clients. Masquerade (NAT) enabled for outbound Internet access. All outbound traffic from the LAN is allowed. |
| `vpn` | WireGuard (`wg0`) | WireGuard tunnel traffic | Semi-trusted. Firewall rules control which internal services VPN peers can reach. Traffic to the LAN is restricted to specific services and ports. |
| `vpn-<key>` | WireGuard (per-access-class interfaces) | WireGuard tunnel traffic, per access class | Semi-trusted. Created and maintained automatically by the WireGuard→firewall sync: one zone per access class with peers, with the class's WG interface assigned, masquerade enabled, a UDP listen-port accept rule, and inter-zone accept rules for internal subnets when the class has `lan_access`. A plain `vpn` zone is managed only as a legacy fallback for peers without an access class. |
| `trusted` / `loopback` | `lo` | Localhost communication | unrestricted; used for the Flask-to-nginx management proxy. |
| Custom zones | — | DMZ, guest networks, etc. | Additional zones can be created to isolate specific network segments with their own rule sets. |
### IP Forwarding and NAT
IP forwarding (`net.ipv4.ip_forward = 1`) is enabled system-wide to allow routing between zones (LAN to Internet, VPN to LAN). However, actual traffic flow is controlled by firewalld rules. Masquerade is enabled on the `internal` zone so that LAN clients get NAT translation when accessing the Internet through the Vacuum Wall router.
IP forwarding is **not** auto-enabled by Vacuum Wall — `net.ipv4.ip_forward` is one of the allowlisted sysctl keys an operator can set through the network API, and actual traffic flow is controlled by firewalld rules. Masquerade is auto-enabled by the WireGuard→firewall sync **only on VPN zones** (the per-access-class `vpn-<key>` zones and the legacy `vpn` zone), not on `internal`.
The `public` zone is special-cased around masquerade:
- **Refusal**: the masquerade endpoint refuses to enable masquerade on `public` — masquerade must be enabled on `internal` or `vpn` instead.
- **Auto-propagation**: at apply time, if any non-`public` zone has masquerade enabled, `apply` propagates masquerade to the `public` zone (and removes it when no non-public zone needs it), writing the propagated state back to the declarative config. Under the nftables backend, traffic exiting through a `public`-zoned WAN interface hits `public`'s POSTROUTING chain rather than the internal zone's, so NAT would silently fail without this propagation.
### Management Lockout Guard
The firewalld default zone is the catch-all for unassigned interfaces (normally the WAN), so removing both `https` (management access via nginx) and `ssh` (remote recovery) from it would leave no path back except a physical console. The config apply path and the per-zone services endpoint refuse such a change with HTTP `409` unless the request passes `{"force": true}`. The guard fails closed: if the default zone cannot be determined, the operation is treated as a lockout and refused.
### Interface-Coverage Invariant
Every interface managed by the network subsystem (`lo` and `wg*` excluded) must be covered by a zone in `config/firewall/config.json` or listed under the top-level `unmanaged` key. The config is the source of truth for zone interfaces — an omitted `interfaces` key counts as empty — so the check is computed from the config alone with no live-state fallback. Violations are rejected with HTTP `400` at save time (`POST`/`PATCH /firewall/config`) and HTTP `409` at apply time (`POST /firewall/config/apply`, overridable with `force: true`). Live drift is advisory only (the `uncovered_interfaces` state field).
## Input Validation
+368
View File
@@ -0,0 +1,368 @@
# State Model Reference
Authoritative reference for the shapes returned by the daemon's
pre-computed state store (`lib/state.py`), collected per subsystem and
pushed over the WebSocket (snapshot on connect, per-subsystem deltas
after every change).
Python schemas live in `lib/schema.py` (TypedDicts); each collector's
return annotation references them.
## Shared notes
- Every collector return carries a top-level `timestamp` (ISO-8601).
- Subsystems with a declarative config expose pending state as a status
dict: `status: {"pending_changes": bool, "pending_diff": [...]}`,
**except firewall**, which uses `pending: {config_pending() result}`
(a separate live-drift mechanism, see Firewall below).
- `pending_diff` lists the field-level changes since the last apply;
each entry has the shape:
```
{path: str, action: "added"|"removed"|"changed",
old: <value>|null, new: <value>|null}
```
`path` is a dotted key path; lists of equal length are compared
element-by-element with `[i]` indexes, while any other difference
(including a length change) is reported as a single `changed` entry.
`old` is `null` for added fields, `new` is `null` for removed ones.
Apply-bookkeeping keys (`_last_applied_*`) are ignored. The list is
empty when up to date or when no applied snapshot is recorded.
- A subsystem whose collection failed holds `null`/`None` in the state
store. Null handling differs per push layer:
- **snapshot** is NOT filtered server-side — `get_snapshot()` is sent
verbatim, including `null` entries; the client skips `null` payloads
so a failed collector never overwrites good client data.
- **versions** deltas ARE filtered server-side — the daemon skips the
broadcast when the subsystem data is `null`.
- **tick** has no `None` guard (it cannot be `null` in practice: a
tick is only broadcast after a successful poll).
- A **poll failure** does NOT set state to `null` — the stale value is
retained and no broadcast is sent. Only `populate()` (startup and
mutation-triggered refreshes) clears a subsystem to `null` when its
collection fails.
- Config-backed subsystems record their applied baseline inside the config
file itself: `_last_applied_config` (the full merged config at last
apply) and `_last_applied_hash` (its SHA-256). A hash subsystem's
`status.pending_changes` is true when the current (merged) config hash
differs from the recorded hash — **or when no hash is recorded at all**
(the config was never applied). All apply operations (including
firewall `config_apply`) re-stamp the baseline. These bookkeeping keys
are internal and stripped from every state/API config payload.
Canceling pending changes (`POST /api/status/cancel-all`) restores a
pending config file from its snapshot; a subsystem with no recorded
baseline (never applied) is reported as skipped, not reset. Apply-all
and cancel-all operate on freshly re-collected state, not last-poll
state: every mutation ends with `emit_and_refresh()` → a synchronous
`refresh_state()` re-collection, so a saved edit is already reflected
by the time either endpoint runs; only out-of-band changes (e.g. manual
edits) can lag the poll interval. Cancel reverts only the declarative
config file — live drift (e.g. manual `firewall-cmd`) survives a cancel.
## State shape summary
`state_store.get(<subsystem>)` returns:
| Subsystem | Poll | Volatile fields | Top-level keys |
|---|---|---|---|
| `firewall` | 30s | `interfaces[].ips`, `interfaces[].ipv6` | `config`, `active_zones`, `default_zone`, `interfaces`, `available_services`, `service_descriptions`, `uncovered_interfaces`, `zones`, `rich_rules`, `pending`, `timestamp` |
| `dnsmasq` | 10s | *(none)* | `config`, `status`, `leases`, `timestamp` |
| `nginx` | 60s | *(none)* | `config`, `domains`, `status`, `timestamp` |
| `acme` | 300s | *(none)* | `certs`, `email`, `account`, `status`, `timestamp` |
| `wireguard` | 10s | `status.peers[].transfer_received`/`.transfer_sent`/`.latest_handshake` and the same three under `status.classes[].peers[]` | `config`, `status`, `peers`, `timestamp` |
| `networkd` | 10s | `interfaces[].addresses` | `config`, `interfaces`, `status`, `timestamp` |
| `system` | 1s | `load`, `memory`, `swap`, `traffic` | `load`, `memory`, `swap`, `traffic`, `timestamp` |
Poll intervals are overridable via `VACUUM_WALL_POLL_INTERVALS`
(`subsystem:seconds,subsystem:seconds`).
## Firewall
Top-level `FirewallState`:
```
{
config: {}, // config/firewall/config.json
active_zones: {zone: [iface]}, // zones with assigned interfaces
default_zone: str, // firewall-cmd --get-default-zone;
// catch-all zone for interfaces with
// no explicit assignment
interfaces: [ // ip link/addr parsing
{name, mac, state, mtu, ips, ipv6, zone}
],
available_services: [str], // firewall-cmd --get-services
service_descriptions: {svc: str}, // one-line description from the
// firewalld service XML definitions
// (lib/firewall.py get_service_descriptions,
// cached per process)
uncovered_interfaces: [str], // network-config interfaces (excluding
// lo/wg*) not in any LIVE zone — a
// live-drift advisory (config may still
// cover them); distinct from the
// config-based interface-coverage
// invariant (docs/config.md); NOT
// counted in pending
zones: {zone: zoneDict}, // --list-all-zones; hyphenated keys,
// may carry "sources", "ports",
// "protocols", "forward-ports", "ics",
// "icmp-blocks", "module", "rich-rules"
rich_rules: {zone: [str]}, // raw firewalld rich-rule strings,
// re-derived from zones (NO ids —
// deletion-by-id uses config.zones[].rich_rules)
pending: { // config_pending() (lib/firewall.py)
pending: [...], needs_apply: bool,
unmanaged_zones: {zone: {interfaces: [...]}}
},
timestamp: str,
}
```
Notes:
- `interfaces[].ips` / `interfaces[].ipv6` hold `"ip/prefix"` strings
(IPv6 list is separate).
- The zone dict's rich-rules key is HYPHENATED (`"rich-rules"`);
`state.rich_rules` is the snake_case top-level re-derivation.
- `pending.pending[]` change dicts have one of two shapes (see
"Firewall pending summary" below): `{zone, type, config, live}` or
`{zone, type, config_count, live_count}`.
## Dnsmasq
```
{
config: {}, // config/dnsmasq/config.json, deep-merged
status: {
service_active: bool, config_file_exists: bool,
active_leases: int, pending_changes: bool,
pending_diff: [pending_change] // see Shared notes
},
leases: [
{expires, mac, ip, hostname, interface} // expires = ISO-8601 or ""
],
timestamp: str,
}
```
## Nginx
```
{
config: {}, // config/nginx/config.json
domains: [ // flattened: one entry per domain+path
{domain, path, backend, online, force_ssl, backend_name, cert,
[is_management], [is_websocket]}
],
status: {pending_changes: bool,
pending_diff: [pending_change]},
timestamp: str,
}
```
## ACME
```
{
certs: [ // list_certs(); extra keys possible
{domain, expiry, renewed, status, days_remaining, ...}
],
email: str,
account: {registered, email, ca, key_length},
status: {error: str|null}, // null on success; the cert-collection
// failure message otherwise (certs is
// then [])
timestamp: str,
}
```
Notes:
- `status.error` is the one failure signal: cert collection failed
(e.g. unreadable `account.conf` after an ownership flip). `certs` is
`[]` while the rest of the state is still collected, so a broken
acme.sh does not blank the whole dashboard; the poll diff detects the
recovery when the error clears.
- Before listing, the collector runs a cheap no-sudo **self-heal probe**:
it walks `ACME_HOME` for files that lost their group-read bit (acme.sh
re-hardens its tree to `chmod 600` on every run) and, only when one is
found, re-runs the sudo permission normalization. The steady-state poll
therefore makes no sudo call.
- When the failure text contains an unreadable `account.conf`
(`Permission denied`), the error is rewritten into an actionable
remediation: `sudo chown <daemon-user>:<group> <ACME_HOME>/account.conf
&& sudo chmod 0640 <ACME_HOME>/account.conf`, then restart
`vacuum-walld`.
## WireGuard
```
{
config: {}, // private_key stripped from interface
// AND every access class
status: {
up: bool, // true when ANY managed iface is up
interface: {}, peers: [], // legacy single interface (wg0)
classes: {class: {up, interface, peers}}, // per wg-<class>
pending_changes: bool,
pending_diff: [pending_change] // entries whose path contains
// "private_key" are dropped, so the
// diff never exposes key material
},
peers: [ // config peers, private keys stripped
{name, public_key, endpoint, allowed_ips,
persistent_keepalive, preshared_key, ...}
],
timestamp: str,
}
```
Runtime peers (`status.peers[]`, `status.classes[].peers[]`) carry:
`public_key`, `endpoint`, `allowed_ips`, `latest_handshake`,
`transfer_received`, `transfer_sent`, `persistent_keepalive`.
## Networkd
Matches `parse_networkctl_status()` output (lib/network.py) exactly:
```
{
config: {}, // config/network/config.json
interfaces: {iface: { // flat runtime entry per interface;
addresses: ["ip/prefix"], // a single combined addresses list
gateway, dns: [str], mac, // (no ipv6_addresses/routes keys)
state, link}
},
status: {pending_changes: bool,
pending_diff: [pending_change]},
timestamp: str,
}
```
The parser does not filter `lo`; clients that don't want it filter
client-side.
## System
Metrics only — no config, no pending state.
```
{
load: {load1, load5, load15},
memory: {total, available, used, used_pct}, // bytes; 0-100
swap: {total, used, used_pct}, // bytes; 0-100
traffic: {iface: {rx_bytes, tx_bytes,
rx_packets, tx_packets}},
timestamp: str,
}
```
All four metric fields (`load`, `memory`, `swap`, and the whole
`traffic` dict) are volatile, and `timestamp` is excluded from both diff
layers — so a **structural diff can never fire for `system`**. After the
first populate (which always counts as structural and broadcasts a
`versions` envelope), every change is a `tick`.
## Firewall pending summary
`GET /api/firewall/config/pending` returns the state's `pending` dict
plus `pending_summary` — a list of human-readable strings, one per
pending change. Each firewall pending change has one of two shapes:
```
{zone: str, type: str, config: <value>, live: <value>}
// type ∈ {interfaces, services, masquerade, target}
{zone: str, type: str, config_count: int, live_count: int}
// type ∈ {rich_rules, forward_ports}
```
## Apply-all / cancel-all API
All endpoints are proxied to the daemon (`daemon/handlers/status.py`).
Subsystems are processed in dependency order
`SYS_ORDER = ["networkd", "firewall", "wireguard", "dnsmasq", "nginx"]`.
- `GET /api/status/pending` — aggregated pending state:
```
{
firewall: {
needs_apply: bool,
change_count: int,
changes: [{summary: str, detail: ""}],
uncovered_interfaces: [str], // advisory — never counted
coverage_warnings: [str] // advisory — never counted
},
dnsmasq: {pending_changes: bool, summary: str,
changes: [{summary, detail}]},
nginx: {…same…},
wireguard: {…same…},
networkd: {…same…},
total_changes: int,
}
```
- `POST /api/status/apply-all` — applies **only the pending**
subsystems, in `SYS_ORDER`. Body `{"force": true}` is forwarded to the
firewall apply only (it overrides the firewall's management-lockout and
interface-coverage guards; other subsystems ignore it). Response:
`{applied: [subsystem], errors: {label: msg}}`.
- `POST /api/status/cancel-all` — reverts **only the pending**
subsystems' config files to their last-applied snapshot (no
live-system commands run). Response: `{cancelled: [subsystem],
skipped: {label: reason}, errors: {label: msg}}` — `skipped` covers
e.g. "No baseline recorded (never applied)".
- `POST /api/status/refresh` — re-collect state and return the snapshot
for the target subsystems; optional body `{"subsystems": [name, …]}`
filter (all when omitted). **No version bump** — versions advance on
structural poll diffs and mutation-triggered refreshes only.
## Diff & push mechanics
Envelope shapes (daemon → client):
```
{"type": "snapshot", "data": {subsystem: state|null, …}} // on connect
{"type": "versions", "subsystem": str, "data": state} // structural
{"type": "tick", "subsystem": str, "data": state} // volatile
```
- **First poll**: when the previous state is `null` (not yet populated),
the poll counts as structural — the first broadcast after startup is a
`versions` envelope.
- **Two-layer diff** (`lib.state._diff_layers`):
- structural layer — volatile fields zeroed out, `timestamp` removed;
- volatile layer — full data minus `timestamp`, computed only when the
structural layer is unchanged.
- When the structural layer changes, the volatile signal is
**suppressed** (reported unchanged): the `versions` envelope already
carries the full new data, so a tick would be redundant.
- `timestamp` is excluded from **both** layers — a timestamp-only
change never triggers either envelope.
- **Version bumps**: structural polls and mutation-triggered refreshes
(`refresh_state`, default `bump=True`) bump the subsystem version
counter; `tick` broadcasts and `POST /api/status/refresh` never bump.
The counter is not sent over the wire — the envelope itself is the
signal.
- **Poll failure** = no broadcast (stale state retained; see Shared
notes).
- **Client mapping**: `networkd` maps to the `network` model
(`_SUBSYSTEM_TO_MODEL` in `websocket.js`); unknown or retired message
types are ignored.
- **HTTP fallback**: if the WS snapshot has not populated a model within
3 s of page load, the client fetches over HTTP instead —
`POST /api/status/refresh` with `{"subsystems": [name]}` returns the
subsystem state verbatim; a `null` payload fails the fetch and the
model keeps its schema defaults.
- **Interval overrides**: `VACUUM_WALL_POLL_INTERVALS`
(`subsystem:seconds,subsystem:seconds`) is parsed at daemon startup;
entries whose value is `<= 0` or not an integer are skipped with a
logged warning (the subsystem keeps its default interval).
## Frontend schema defaults (stale — follow-up)
`webui/static/hoover/schema.js` holds hand-maintained `defaults` for
every state model (placeholder data before the first WS snapshot / HTTP
fetch). They are currently **stale copies** of this reference: no
firewall `default_zone`, no acme `status`, no `pending_diff` keys. Since
they only seed initial model data and are replaced verbatim by the first
real payload, this is a cosmetic gap — flagged for follow-up (a code
change, not a doc change).
+100 -6
View File
@@ -32,6 +32,11 @@ _ACME_ENVIRON = {
_WEBROOT = PROJECT_DIR / "data" / "acme" / "www"
def get_acme_home() -> Path:
"""Resolve the ACME home directory (``ACME_HOME`` env, default ``data/acme``)."""
return Path(os.environ.get("ACME_HOME", str(_ACME_HOME)))
def _find_acme() -> str:
"""Locate the acme.sh binary on the system.
@@ -87,7 +92,7 @@ def _run_acme(args: list[str]) -> str:
acme_bin = _find_acme()
# Check for ACME_HOME env var (set by systemd in production)
acme_home_env = os.environ.get("ACME_HOME", str(_ACME_HOME))
acme_home_env = str(get_acme_home())
cmd: list[str] = [
acme_bin,
@@ -96,6 +101,17 @@ def _run_acme(args: list[str]) -> str:
"--config-home",
acme_home_env,
*args,
# Append the full transcript to $ACME_HOME/acme.sh.log so manual
# runs (whose stdout is captured below) leave a persistent record
# of the raw CA exchange. The log file is passed explicitly (never
# as a bare trailing --log): a valueless trailing --log makes
# acme.sh's arg loop double-shift under dash (the --log branch
# shifts once, then the loop's trailing `shift 1` runs with zero
# positional params) and fails with "shift: can't shift that many"
# (exit 2). The explicit path keeps the same default destination
# ($LE_CONFIG_HOME/acme.sh.log) and can never swallow a real arg.
"--log",
str(Path(acme_home_env) / "acme.sh.log"),
]
try:
@@ -118,12 +134,38 @@ def _run_acme(args: list[str]) -> str:
if result.returncode != 0:
logger.error("acme.sh failed (rc=%d): %s", result.returncode, output.strip())
raise RuntimeError(
f"acme.sh failed with exit code {result.returncode}: {output.strip()}"
f"acme.sh failed with exit code {result.returncode}: "
f"{_summarize_acme_output(output)}"
)
return output
def _summarize_acme_output(output: str) -> str:
"""Reduce raw acme.sh output to a concise, human-readable summary.
acme.sh prints timestamped transcript lines; the failure reason is
in the final lines (e.g. "The retryafter=86400 value is too large
(> 600), will not retry anymore."). Strips per-line timestamps and
the "Please check log file" pointer so the summary stays toast-
sized. A "Permission denied" diagnostic is preserved even when it
is not among the final lines the actionable-error matcher in
daemon/collectors/acme.py keys off it. The full transcript remains
in the log and acme.sh.log.
"""
lines = [line.strip() for line in output.strip().splitlines() if line.strip()]
lines = [re.sub(r"^\[[^\]]*\] ", "", line) for line in lines]
lines = [line for line in lines if not line.startswith("Please check log file")]
if not lines:
return "(no output)"
tail = list(lines[-2:])
for line in reversed(lines):
if "Permission denied" in line and line not in tail:
tail.insert(0, line)
break
return "; ".join(tail)
def set_email(email: str) -> None:
"""Configure the default ACME contact email.
@@ -248,7 +290,7 @@ def list_certs() -> list[dict]:
A list of dicts, one per certificate, with keys matching
the cert-info schema (domain, ca, cert_path, etc.).
"""
raw = _run_acme(["--list"])
raw = _run_acme(["--list", "--listraw"])
certs: list[dict] = []
entries = _parse_list_output(raw)
@@ -472,7 +514,7 @@ def deploy(domain: str) -> None:
def _split_line(line: str, separator: str | None) -> list[str]:
"""Split a line by *separator*, falling back to whitespace for column output."""
if separator in line:
if separator is not None and separator in line:
return line.split(separator)
return line.split()
@@ -503,8 +545,8 @@ def _parse_list_output(raw: str) -> list[dict]:
headers = _split_line(header_line, "\t")
separator = "\t"
else:
headers = _split_line(header_line, None) # whitespace
separator = None
# Column-aligned: use position-based parsing via helper
return _parse_column_aligned(header_line, lines[1:])
if "Main_Domain" not in headers:
raise ValueError(
@@ -526,6 +568,57 @@ def _parse_list_output(raw: str) -> list[dict]:
return entries
def _find_header_positions(header_line: str):
"""Find start positions of each header word in a column-aligned header."""
names: list[str] = []
starts: list[int] = []
i = 0
while i < len(header_line):
while i < len(header_line) and header_line[i] == " ":
i += 1
j = i
while j < len(header_line) and header_line[j] != " ":
j += 1
if j > i:
names.append(header_line[i:j])
starts.append(i)
i = j
return names, starts
def _parse_column_aligned(header_line: str, data_lines: list[str]) -> list[dict]:
"""Parse column-aligned output using header positions to locate fields.
Unlike simple whitespace splitting, this preserves empty fields by using
character positions rather than token counts. Empty columns (e.g. missing
Profile or SAN_Domains) are correctly handled.
"""
names, starts = _find_header_positions(header_line)
if "Main_Domain" not in names:
raise ValueError(
f"acme.sh --list output is not in expected format: {header_line!r}"
)
# Column ends: midpoint before next header starts (or end of line for last)
ends: list[int] = len(starts) * [len(header_line)]
for i in range(len(starts) - 1):
ends[i] = (starts[i] + starts[i + 1]) // 2
entries: list[dict] = []
for line in data_lines:
line = line.rstrip()
if not line.strip():
continue
entry: dict[str, str] = {}
for k, name in enumerate(names):
s, e = starts[k], ends[k]
cell = line[s:e] if len(line) > s else ""
entry[name.lower()] = cell.strip().strip('"')
entries.append(entry)
return entries
def _days_until(date_str: str) -> int | None:
"""Parse an ISO date string and return days until that date from now."""
if not date_str:
@@ -569,6 +662,7 @@ __all__ = [
"days_until_expiry",
"deploy",
"find_cert_dir",
"get_acme_home",
"get_cert_info",
"get_cert_paths",
"get_email",
+467
View File
@@ -0,0 +1,467 @@
"""JWT authentication module for Vacuum Wall.
Handles token creation, validation, refresh, and blacklisting.
Each user has their own JWT signing secret stored in the database.
Configuration comes from config/auth/config.json.
"""
from __future__ import annotations
import base64
import json
import logging
import random
import secrets
import time
import uuid
from pathlib import Path
from typing import Any
import jwt
from lib.common import load_json
from lib.db import (
Q_DELETE_EXPIRED_BLACKLIST,
Q_DELETE_REFRESH_TOKEN,
Q_INSERT_BLACKLIST,
Q_SELECT_BLACKLIST,
Q_SELECT_REFRESH_TOKEN,
Q_SELECT_USER_JWT_SECRET,
Q_UPDATE_JWT_SECRET,
Q_UPSERT_REFRESH_TOKEN,
get_db,
)
logger = logging.getLogger(__name__)
AUTH_CONFIG_PATH = (
Path(__file__).resolve().parent.parent / "config" / "auth" / "config.json"
)
_DEFAULT_JWT_CONFIG = {
"access_token_ttl": 900,
"refresh_token_ttl": 604800,
"algorithm": "HS256",
}
def _get_jwt_config() -> dict[str, Any]:
"""Load JWT configuration from auth config."""
raw = load_json(AUTH_CONFIG_PATH)
return raw.get("jwt", _DEFAULT_JWT_CONFIG)
def get_user_jwt_secret(username: str) -> str | None:
"""Return the JWT signing secret for *username*, or ``None`` if not found."""
db = get_db()
rows = db.query(Q_SELECT_USER_JWT_SECRET, (username,))
if not rows:
return None
return rows[0]["jwt_secret"]
def rotate_user_secret(username: str) -> None:
"""Rotate the JWT secret for *username*, invalidating all their existing tokens.
Used when a user's password is changed to ensure all prior sessions
are immediately terminated regardless of token expiration.
"""
new_secret = secrets.token_urlsafe(32)
db = get_db()
db.run(Q_UPDATE_JWT_SECRET, (new_secret, username))
logger.warning(
"JWT secret rotated for %r — their existing tokens are now invalid", username
)
def get_access_ttl() -> int:
"""Return access token TTL in seconds."""
return _get_jwt_config().get("access_token_ttl", 900)
def get_refresh_ttl() -> int:
"""Return refresh token TTL in seconds."""
return _get_jwt_config().get("refresh_token_ttl", 604800)
def get_algorithm() -> str:
"""Return the JWT algorithm."""
return _get_jwt_config().get("algorithm", "HS256")
def generate_access_token(
username: str,
permissions: dict[str, str],
session_id: str | None = None,
) -> str:
"""Generate a new access token for *username*.
Args:
username: The authenticated username.
permissions: Dict mapping subsystem names to permission levels.
session_id: Optional session binding ID. Included in the token payload
so the Flask middleware can tie the token to the browser session
that created it.
Returns:
JWT token string.
Raises:
RuntimeError: If JWT secret is not configured.
"""
secret = get_user_jwt_secret(username)
if not secret:
raise RuntimeError(f"JWT secret not configured for user {username!r}")
algorithm = get_algorithm()
now = int(time.time())
payload = {
"sub": username,
"exp": now + get_access_ttl(),
"iat": now,
"jti": str(uuid.uuid4()),
"type": "access",
"permissions": permissions,
"session_id": session_id or secrets.token_urlsafe(16),
}
return jwt.encode(payload, secret, algorithm=algorithm)
def generate_refresh_token(username: str, session_id: str | None = None) -> str:
"""Generate a new refresh token for *username*.
Args:
username: The authenticated username.
session_id: Session binding ID included in the token payload.
When present, the refresh endpoint requires a matching session_id,
preventing a stolen refresh token from being usable without the
originating browser session.
Returns:
JWT refresh token string.
Raises:
RuntimeError: If JWT secret is not configured.
"""
secret = get_user_jwt_secret(username)
if not secret:
raise RuntimeError(f"JWT secret not configured for user {username!r}")
algorithm = get_algorithm()
now = int(time.time())
payload = {
"sub": username,
"exp": now + get_refresh_ttl(),
"iat": now,
"jti": str(uuid.uuid4()),
"type": "refresh",
}
if session_id:
payload["session_id"] = session_id
return jwt.encode(payload, secret, algorithm=algorithm)
def generate_tokens(username: str, permissions: dict[str, str]) -> dict[str, str]:
"""Generate both access and refresh tokens.
Args:
username: The authenticated username.
permissions: Dict mapping subsystem names to permission levels.
Returns:
Dict with ``access_token`` and ``refresh_token`` keys.
"""
session_id = secrets.token_urlsafe(16)
access_token = generate_access_token(username, permissions, session_id)
refresh_token = generate_refresh_token(username, session_id)
_persist_refresh_token(username, refresh_token)
return {
"access_token": access_token,
"refresh_token": refresh_token,
"session_id": session_id,
}
def _persist_refresh_token(username: str, refresh_token: str) -> None:
"""Persist the active refresh token JTI for *username* in the database.
One row per user replaces any existing entry on upsert.
Args:
username: The username.
refresh_token: The JWT refresh token string.
"""
payload = decode_token(refresh_token)
if payload is None:
return
jti = payload.get("jti")
if not jti:
return
issued_at = payload.get("iat", int(time.time()))
db = get_db()
db.run(Q_UPSERT_REFRESH_TOKEN, (username, jti, issued_at))
def blacklist_active_refresh_token(username: str) -> None:
"""Blacklist the user's currently active refresh token from the database.
Looks up the stored JTI for *username*, blacklists it, and removes the
database entry. Safe to call when no token is registered the query
will simply return no rows.
Args:
username: The username.
"""
db = get_db()
rows = db.query(Q_SELECT_REFRESH_TOKEN, (username,))
if rows:
blacklist_token(rows[0]["jti"], token_type="refresh")
db.run(Q_DELETE_REFRESH_TOKEN, (username,))
def _extract_unverified_sub(token_string: str) -> str | None:
"""Extract the ``sub`` claim from a JWT payload without signature verification.
The JWT payload is the second segment (dot-separated), base64url-encoded JSON.
This is safe because we are NOT trusting the claim value we use it solely
to look up the user's secret for proper verification.
Args:
token_string: The JWT token string.
Returns:
The ``sub`` claim value, or ``None`` if the token is malformed.
"""
try:
parts = token_string.split(".")
if len(parts) != 3:
return None
payload_b64 = parts[1]
# Add padding
padding = 4 - len(payload_b64) % 4
if padding != 4:
payload_b64 += "=" * padding
payload_json = base64.urlsafe_b64decode(payload_b64)
payload = json.loads(payload_json)
if not isinstance(payload, dict):
return None
return payload.get("sub")
except (ValueError, json.JSONDecodeError, UnicodeDecodeError):
return None
def decode_token(token_string: str) -> dict[str, Any] | None:
"""Decode and validate a JWT token using the user's secret.
Extracts the ``sub`` claim from the unverified payload to look up the
correct per-user signing secret, then verifies the signature.
Args:
token_string: The JWT token string (without Bearer prefix).
Returns:
Payload dict if valid, None if invalid/expired or user not found.
"""
sub = _extract_unverified_sub(token_string)
if not sub:
return None
secret = get_user_jwt_secret(sub)
if not secret:
return None
algorithm = get_algorithm()
try:
payload = jwt.decode(token_string, secret, algorithms=[algorithm])
return payload
except (jwt.ExpiredSignatureError, jwt.InvalidTokenError):
return None
def validate_token(
token_string: str,
token_type: str = "access",
session_id: str | None = None,
require_session: bool = False,
) -> dict[str, Any] | None:
"""Validate a JWT token and check it against the blacklist.
Args:
token_string: The JWT token string.
token_type: Expected token type ("access" or "refresh").
session_id: Must match the ``session_id`` claim in the token payload.
When provided, enforces session binding to prevent a stolen token
from being usable without the originating session. When ``None``,
the check is skipped unless ``require_session`` is True.
require_session: If True and the token payload contains a ``session_id``,
the request must provide a matching ``session_id``. Used by the
refresh handler to prevent session binding bypass. When False
(default), omitting ``session_id`` is acceptable even if the token
contains one.
Returns:
Payload dict including permissions, or None if invalid/blacklisted.
"""
payload = decode_token(token_string)
if payload is None:
return None
if payload.get("type") != token_type:
return None
token_session_id = payload.get("session_id")
if require_session and token_session_id is not None:
if session_id is None or session_id != token_session_id:
return None
elif session_id is not None and session_id != token_session_id:
return None
jti = payload.get("jti")
if jti and is_blacklisted(jti):
return None
return payload
def blacklist_token(jti: str, token_type: str = "access") -> None:
"""Add a JTI to the blacklist.
Args:
jti: Token UUID to blacklist.
token_type: Token type ("access" or "refresh"). Defaults to "access".
The blacklist expiry is set to current time + the token type's TTL.
"""
db = get_db()
ttl = get_refresh_ttl() if token_type == "refresh" else get_access_ttl()
db.run(Q_INSERT_BLACKLIST, (jti, token_type, int(time.time()) + ttl))
if random.random() < 0.02:
blacklist_expired()
def is_blacklisted(jti: str) -> bool:
"""Check if a JTI is blacklisted.
Args:
jti: Token UUID to check.
Returns:
True if the token has been blacklisted.
"""
db = get_db()
rows = db.query(Q_SELECT_BLACKLIST, (jti,))
return len(rows) > 0
def blacklist_expired() -> None:
"""Remove expired entries from the blacklist."""
db = get_db()
now = int(time.time())
db.run(Q_DELETE_EXPIRED_BLACKLIST, (now,))
class RateLimiter:
"""Sliding-window rate limiter that tracks successes and failures separately.
Failures are counted against the limit. A successful operation resets
the failure counter for that key.
"""
def __init__(self, max_attempts: int = 5, window_seconds: int = 300) -> None:
self.max_attempts = max_attempts
self.window = window_seconds
self.failures: dict[str, list[float]] = {}
def is_allowed(self, key: str) -> bool:
"""Check if a request from *key* is allowed (does NOT record the attempt).
Args:
key: Identifier for the rate limit bucket (e.g., username or IP).
Returns:
True if the request is allowed, False if rate limited.
"""
now = time.time()
cutoff = now - self.window
timestamps = self.failures.get(key, [])
clean = [t for t in timestamps if t > cutoff]
return len(clean) < self.max_attempts
def record_failure(self, key: str) -> None:
"""Record a failed attempt for *key*."""
self.failures.setdefault(key, []).append(time.time())
def record_success(self, key: str) -> None:
"""Reset the failure counter for *key* on a successful operation."""
self.failures.pop(key, None)
def cleanup(self) -> None:
"""Remove expired entries from all buckets."""
now = time.time()
cutoff = now - self.window
for key in list(self.failures):
self.failures[key] = [t for t in self.failures[key] if t > cutoff]
if not self.failures[key]:
del self.failures[key]
# Global rate limiters — in-memory only. Counts reset on daemon restart
# (SIGHUP reload, process restart). Acceptable for a single-user appliance
# where restarts are rare; brute-force windows briefly reset post-restart.
_login_limiter = RateLimiter(max_attempts=10, window_seconds=300)
_webauthn_limiter = RateLimiter(max_attempts=5, window_seconds=600)
def check_login_rate(username: str, client_ip: str | None = None) -> bool:
"""Check if login is rate-limited for the given username.
Checks failure counts for both IP and username buckets without
recording anything. Callers must invoke record_login_failure() or
record_login_success() after the password verification step.
Args:
username: The login attempt username.
client_ip: The client IP address (from X-Real-IP header).
Returns:
True if the attempt is allowed, False if rate limited.
"""
if client_ip and not _login_limiter.is_allowed(client_ip):
return False
return _login_limiter.is_allowed(username)
def record_login_failure(username: str, client_ip: str | None = None) -> None:
"""Record a failed login attempt."""
if client_ip:
_login_limiter.record_failure(client_ip)
_login_limiter.record_failure(username)
def record_login_success(username: str, client_ip: str | None = None) -> None:
"""Record a successful login (resets failure counter)."""
if client_ip:
_login_limiter.record_success(client_ip)
_login_limiter.record_success(username)
def check_webauthn_rate(username: str, client_ip: str | None = None) -> bool:
"""Check if WebAuthn authentication is rate-limited for the given username.
Args:
username: The WebAuthn attempt username.
client_ip: The client IP address (from X-Real-IP header).
Returns:
True if the attempt is allowed, False if rate limited.
"""
if client_ip and not _webauthn_limiter.is_allowed(client_ip):
return False
return _webauthn_limiter.is_allowed(username)
def record_webauthn_failure(username: str, client_ip: str | None = None) -> None:
"""Record a failed WebAuthn attempt."""
if client_ip:
_webauthn_limiter.record_failure(client_ip)
_webauthn_limiter.record_failure(username)
def record_webauthn_success(username: str, client_ip: str | None = None) -> None:
"""Record a successful WebAuthn attempt (resets failure counter)."""
if client_ip:
_webauthn_limiter.record_success(client_ip)
_webauthn_limiter.record_success(username)
+324
View File
@@ -0,0 +1,324 @@
"""User CRUD and permission management for Vacuum Wall authentication.
All operations use DB query IDs through the Database abstract layer.
Password hashing uses Argon2id via lib.password.
"""
from __future__ import annotations
import logging
import re
import secrets
from typing import Any
from lib.auth import (
blacklist_active_refresh_token,
rotate_user_secret,
)
from lib.db import (
Q_DELETE_PERMISSION_SUBSYSTEM,
Q_DELETE_USER,
Q_INSERT_USER,
Q_SELECT_PERMISSIONS,
Q_SELECT_USER_BY_NAME,
Q_SELECT_USERS_WITH_PERMS,
Q_UPDATE_PASSWORD,
Q_UPSERT_PERMISSION,
get_db,
)
from lib.password import hash_password, needs_rehash, verify_password
logger = logging.getLogger(__name__)
# Pre-computed dummy hash for constant-time verification on nonexistent users.
# Generated once at module load to avoid timing leaks from per-call hash generation.
_DUMMY_HASH = hash_password(secrets.token_hex(32))
# Builtin admin — hardcoded, full access, cannot be modified/deleted
BUILTIN_ADMIN_USERNAME = "admin"
# All subsystem names for default permission assignment
ALL_SUBSYSTEMS = [
"firewall",
"network",
"dhcp",
"proxy",
"certs",
"wireguard",
"logs",
"status",
"auth",
]
_USERNAME_RE = re.compile(r"^[a-zA-Z0-9_-]{3,32}$")
def _get_permissions(username: str) -> dict[str, str]:
"""Load permissions for *username* from the database.
Args:
username: The username to load permissions for.
Returns:
Dict mapping subsystem names to permission levels.
"""
db = get_db()
rows = db.query(Q_SELECT_PERMISSIONS, (username,))
return {row["subsystem"]: row["level"] for row in rows}
def get_user(username: str) -> dict[str, Any] | None:
"""Get a user by username (without password hash or JWT secret).
Args:
username: The username to look up.
Returns:
User dict with id, username, permissions, or None if not found.
"""
user = find_user(username)
if user is None:
return None
return {
"id": user["id"],
"username": user["username"],
"permissions": _get_permissions(username),
}
def find_user(username: str) -> dict[str, Any] | None:
"""Find a user by username, including password hash and JWT secret.
Used for password verification and token operations. Not returned through APIs.
Args:
username: The username to look up.
Returns:
User dict from the database row, or None.
"""
db = get_db()
rows = db.query(Q_SELECT_USER_BY_NAME, (username,))
if not rows:
return None
return rows[0]
def verify_user_password(username: str, password: str) -> dict[str, Any] | None:
"""Verify a user's password using Argon2id.
Args:
username: The username to verify.
password: Plain-text password.
Returns:
User dict (without password hash) if the password is correct, None otherwise.
"""
user = find_user(username)
if user is None:
# Run a dummy Argon2id verification against a pre-computed hash to
# prevent timing-based user enumeration. Uses module-level hash
# so both paths take ~1 verify call (~200ms) instead of ~400ms.
verify_password(password, _DUMMY_HASH)
return None
if not verify_password(password, user["password_hash"]):
return None
if needs_rehash(user["password_hash"]):
new_hash = hash_password(password)
db = get_db()
db.run(Q_UPDATE_PASSWORD, (new_hash, username))
logger.info("Password hash rehashed for %r (param upgrade)", username)
return {
"id": user["id"],
"username": user["username"],
"permissions": _get_permissions(username),
}
def create_user(
username: str,
password: str,
permissions: dict[str, str] | None = None,
) -> dict[str, Any]:
"""Create a new user with the given password and permissions.
Args:
username: The new username (3-32 alphanumeric chars, dash, underscore).
password: Plain-text password that will be hashed with Argon2id.
permissions: Dict mapping subsystem names to "read" or "rw".
Returns:
The created user dict with id, username, permissions.
Raises:
ValueError: If username is invalid or already exists.
"""
if not _USERNAME_RE.match(username):
raise ValueError(
"Username must be 3-32 characters: letters, digits, dash, underscore"
)
existing = find_user(username)
if existing is not None:
raise ValueError(f"User {username!r} already exists")
password_hash = hash_password(password)
jwt_secret = secrets.token_urlsafe(32)
db = get_db()
with db.in_transaction() as tx:
user_id = tx.run_one(Q_INSERT_USER, (username, password_hash, jwt_secret))
if permissions:
for subsystem, level in permissions.items():
tx.run(Q_UPSERT_PERMISSION, (username, subsystem, level))
return {
"id": user_id,
"username": username,
"permissions": _get_permissions(username),
}
def update_password(username: str, old_password: str, new_password: str) -> bool:
"""Update a user's password and invalidate all active tokens.
Rotates the user's JWT secret, immediately invalidating all existing
access and refresh tokens.
Args:
username: The username.
old_password: Current password.
new_password: New plain-text password.
Returns:
True if password was updated.
Raises:
ValueError: If old password is incorrect.
"""
if not verify_user_password(username, old_password):
raise ValueError("Current password is incorrect")
blacklist_active_refresh_token(username)
new_hash = hash_password(new_password)
rotate_user_secret(username)
db = get_db()
db.run(Q_UPDATE_PASSWORD, (new_hash, username))
return True
def reset_password(username: str, new_password: str) -> None:
"""Force-reset a user's password without verifying the old one.
Non-interactive variant for install-time and lockout recovery: the
installer does not know the previous password by construction. Rotates
the user's JWT secret and blacklists the active refresh token,
invalidating all existing sessions.
Args:
username: The user to reset.
new_password: New plain-text password.
Raises:
ValueError: If the user does not exist.
"""
if find_user(username) is None:
raise ValueError(f"User {username!r} not found")
blacklist_active_refresh_token(username)
new_hash = hash_password(new_password)
rotate_user_secret(username)
db = get_db()
db.run(Q_UPDATE_PASSWORD, (new_hash, username))
def update_permissions(username: str, permissions: dict[str, str]) -> None:
"""Update a user's permissions and invalidate all existing tokens.
Replaces all existing permissions with the provided mapping. Rotates the
JWT secret so that permission changes take effect immediately existing
tokens with stale permissions are no longer valid.
Args:
username: The username.
permissions: Dict mapping subsystem names to permission levels.
Raises:
ValueError: If attempting to modify builtin admin.
"""
if username == BUILTIN_ADMIN_USERNAME:
raise ValueError("Cannot modify permissions for builtin admin")
user = find_user(username)
if user is None:
raise ValueError(f"User {username!r} not found")
db = get_db()
with db.in_transaction() as tx:
# Upsert all new permissions first, then remove stale ones.
# This order ensures that if an upsert fails mid-loop, the user's
# permissions remain intact (transaction rolls back) rather than
# being permanently wiped.
existing = {
row["subsystem"]: row["level"]
for row in db.query(Q_SELECT_PERMISSIONS, (username,))
}
for subsystem, level in permissions.items():
tx.run(Q_UPSERT_PERMISSION, (username, subsystem, level))
for subsystem in existing:
if subsystem not in permissions:
tx.run(Q_DELETE_PERMISSION_SUBSYSTEM, (username, subsystem))
blacklist_active_refresh_token(username)
rotate_user_secret(username)
def list_users() -> list[dict[str, Any]]:
"""List all users with their permissions.
Returns:
List of user summary dicts.
"""
db = get_db()
rows = db.query(Q_SELECT_USERS_WITH_PERMS, ())
users: dict[int, dict[str, Any]] = {}
for row in rows:
uid = row["id"]
if uid not in users:
users[uid] = {
"id": uid,
"username": row["username"],
"permissions": {},
"created_at": row["created_at"],
}
if row["subsystem"] is not None:
users[uid]["permissions"][row["subsystem"]] = row["level"]
return list(users.values())
def delete_user(username: str) -> bool:
"""Delete a user and all their associated data.
Permissions and WebAuthn credentials are CASCADE-deleted by the schema.
Args:
username: The username to delete.
Returns:
True if the user was deleted.
Raises:
ValueError: If the user does not exist or is the builtin admin.
"""
if username == BUILTIN_ADMIN_USERNAME:
raise ValueError("Cannot delete builtin admin user")
user = find_user(username)
if user is None:
raise ValueError(f"User {username!r} not found")
blacklist_active_refresh_token(username)
db = get_db()
db.run(Q_DELETE_USER, (username,))
return True
+40
View File
@@ -0,0 +1,40 @@
"""Daemon-startup filesystem bootstrap.
Runs once at daemon startup, after the system-config import and before the
first state collection. Creates the runtime directories subsystems
read/write and persists the one-shot nginx legacy-format migration.
Config *files* are deliberately NOT created here: ``get_config`` reads are
pure and return in-memory defaults, and the system-config import must see
absent files in order to adopt live system state on first start. Files are
materialized on the first ``save_config`` (or by the import itself).
"""
from lib import dnsmasq, firewall, network, nginx, wireguard
from lib.common import ensure_dirs
__all__ = ["bootstrap"]
def bootstrap() -> None:
"""Create runtime directories and persist the one-shot nginx migration.
Idempotent existing directories are left untouched and the nginx
migration only rewrites the on-disk file when it actually changes.
"""
ensure_dirs(
dnsmasq.CONFIG_DIR,
dnsmasq.DATA_DIR,
dnsmasq.FRAGMENTS_DIR,
firewall.CONFIG_DIR,
firewall.DATA_DIR,
network.CONFIG_DIR,
network.DATA_DIR,
nginx.CONFIG_DIR,
nginx.SITES_DIR,
wireguard.CONFIG_PATH.parent,
)
# One-shot legacy-format migration for the nginx config (see
# ``lib.nginx.get_config``). Runs here, at startup, so read paths stay
# side-effect free.
nginx.migrate_config_file()
+172
View File
@@ -4,6 +4,7 @@ Provides common helpers for JSON persistence, subprocess execution,
deep merging, and directory creation used across all subsystem modules.
"""
import hashlib
import json
import os
import re
@@ -12,6 +13,130 @@ from copy import deepcopy
from pathlib import Path
from typing import Any
from passlib.hash import sha256_crypt
_APPLY_HASH_KEY = "_last_applied_hash"
_LAST_APPLIED_CONFIG_KEY = "_last_applied_config"
_APPLY_META_KEYS = (_APPLY_HASH_KEY, _LAST_APPLIED_CONFIG_KEY)
def strip_apply_meta(cfg: dict[str, Any]) -> dict[str, Any]:
"""Return *cfg* without any apply bookkeeping keys.
Strips both the last-applied hash and the last-applied config snapshot
so the returned dict reflects only real configuration.
"""
return {k: v for k, v in cfg.items() if k not in _APPLY_META_KEYS}
def config_hash(cfg: dict[str, Any]) -> str:
"""Compute a SHA-256 hash of *cfg*, ignoring apply bookkeeping keys.
Args:
cfg: Config dict, possibly containing ``_last_applied_hash`` and
``_last_applied_config``.
Returns:
Hex digest of the stripped config JSON.
"""
clean = strip_apply_meta(cfg)
return hashlib.sha256(json.dumps(clean, sort_keys=True).encode()).hexdigest()
def stamp_applied(cfg: dict[str, Any]) -> dict[str, Any]:
"""Record that *cfg* is the last applied configuration.
Writes both the content snapshot (``_last_applied_config``) and its hash
(``_last_applied_hash``) so a later pending check can detect drift and a
diff can report exactly which fields changed.
"""
cfg[_APPLY_HASH_KEY] = config_hash(cfg)
cfg[_LAST_APPLIED_CONFIG_KEY] = strip_apply_meta(cfg)
return cfg
def revert_to_applied(path: Path) -> tuple[bool, str]:
"""Restore the config file at *path* to its last-applied snapshot.
Reads the raw file, and when it records a ``_last_applied_config``
snapshot, rewrites the file from that snapshot (stamped with a fresh
hash so the pending check reports the config as up to date).
Args:
path: Path to the config JSON file to revert.
Returns:
Tuple ``(True, "")`` when the file was restored, or
``(False, reason)`` when it could not be (missing file or no
recorded baseline i.e. the config was never applied).
"""
cfg = load_json(path)
snap = cfg.get(_LAST_APPLIED_CONFIG_KEY)
if not isinstance(snap, dict):
return False, "No baseline recorded (never applied)"
save_json(path, stamp_applied(deepcopy(snap)))
return True, ""
def deep_diff(old: Any, new: Any, prefix: str = "") -> list[dict[str, Any]]:
"""Return a list of field-level changes between two configurations.
Each entry is ``{"path", "action", "old", "new"}`` where *action* is one
of ``"added"``, ``"removed"`` or ``"changed"``. Dicts are recursed with
dotted paths; lists of equal length are compared element-by-element,
while any other value that differs is reported as a single change.
Apply bookkeeping keys are ignored.
"""
if isinstance(old, dict):
old = strip_apply_meta(old)
if isinstance(new, dict):
new = strip_apply_meta(new)
out: list[dict[str, Any]] = []
_diff_nodes(old, new, prefix, out)
return out
def _diff_nodes(old: Any, new: Any, path: str, out: list[dict[str, Any]]) -> None:
"""Recursively collect field-level changes from *old* into *new*."""
if isinstance(old, dict) and isinstance(new, dict):
for key in sorted(set(old) | set(new)):
child = f"{path}.{key}" if path else str(key)
if key in old and key in new:
_diff_nodes(old[key], new[key], child, out)
elif key in old:
out.append(
{"path": child, "action": "removed", "old": old[key], "new": None}
)
else:
out.append(
{"path": child, "action": "added", "old": None, "new": new[key]}
)
return
if isinstance(old, list) and isinstance(new, list) and len(old) == len(new):
for i, (o, n) in enumerate(zip(old, new, strict=True)):
_diff_nodes(o, n, f"{path}[{i}]", out)
return
if old != new:
out.append({"path": path, "action": "changed", "old": old, "new": new})
def compute_pending(cfg: dict[str, Any]) -> tuple[bool, list[dict[str, Any]]]:
"""Return ``(pending_changes, pending_diff)`` from apply bookkeeping keys.
``pending_changes`` is ``True`` when the config was never applied or its
content no longer matches the recorded ``_last_applied_hash``. When
pending and a ``_last_applied_config`` snapshot is recorded, the diff is a
field-level comparison of the snapshot against the current (meta-stripped)
config; otherwise it is empty.
"""
pending = _APPLY_HASH_KEY not in cfg or cfg[_APPLY_HASH_KEY] != config_hash(cfg)
diff: list[dict[str, Any]] = []
if pending:
snap = cfg.get(_LAST_APPLIED_CONFIG_KEY)
if isinstance(snap, dict):
diff = deep_diff(snap, strip_apply_meta(cfg))
return pending, diff
def validate_interface_name(name: str) -> str:
"""Validate a Linux network interface name.
@@ -64,6 +189,7 @@ def run(
full_cmd,
capture_output=True,
text=True,
shell=False,
check=check,
timeout=timeout,
)
@@ -99,6 +225,7 @@ def run_proc(
full_cmd,
capture_output=True,
text=True,
shell=False,
check=check,
timeout=timeout,
input=input,
@@ -153,12 +280,57 @@ def ensure_dirs(*dirs: Path) -> None:
d.mkdir(parents=True, exist_ok=True)
def _hash_password(password: str) -> str:
"""Hash *password* using SHA-256 crypt (``$5$`` format) via passlib.
Used for nginx htpasswd files. NOT used for auth user passwords
those use Argon2id via ``lib.password``.
Args:
password: Plain-text password to hash.
Returns:
The hashed password string suitable for ``.htpasswd``
(e.g. ``$5$rounds=$``).
"""
return sha256_crypt.hash(password)
def get_interface_ip(iface: str) -> str | None:
"""Return the primary IPv4 address of *iface* (without CIDR), or ``None``.
Uses ``ip -o addr show`` which is in the daemon sudo whitelist.
"""
if not iface:
return None
try:
raw = run(["ip", "-o", "addr", "show", iface], sudo=True)
for line in raw.splitlines():
parts = line.split()
# -o format: "NUM: IFACE inet/6 ADDR/MASK ..."
if len(parts) >= 4 and parts[2] == "inet":
return parts[3].split("/", 1)[0]
except Exception:
pass
return None
__all__ = [
"_APPLY_HASH_KEY",
"_LAST_APPLIED_CONFIG_KEY",
"_hash_password",
"compute_pending",
"config_hash",
"deep_diff",
"deep_merge",
"ensure_dirs",
"get_interface_ip",
"load_json",
"revert_to_applied",
"run",
"run_proc",
"save_json",
"stamp_applied",
"strip_apply_meta",
"validate_interface_name",
]
+366
View File
@@ -0,0 +1,366 @@
"""Abstract database layer for Vacuum Wall.
Provides query ID constants and an abstract Database baseclass so subsystems
interact with the database through opaque query identifiers, never raw SQL.
Backend implementations (SQLite, PostgreSQL) provide the actual SQL.
Usage:
from lib.db import Q_INSERT_USER, Database, get_db
class SQLiteBackend(Database):
QUERY_MAP = {
Q_INSERT_USER: "INSERT INTO users (username, password_hash) VALUES (?, ?)",
...
}
"""
from __future__ import annotations
import logging
import os
import secrets
import threading
from abc import ABC, abstractmethod
from pathlib import Path
from typing import Any, ClassVar
logger = logging.getLogger(__name__)
PROJECT_DIR = Path(__file__).resolve().parent.parent
# ---------------------------------------------------------------------------
# Query ID constants — single source of truth for all database operations
# ---------------------------------------------------------------------------
Q_INIT_TABLES = "init_tables"
Q_INSERT_USER = "insert_user"
Q_SELECT_USER_BY_NAME = "select_user_by_name"
Q_SELECT_USER_BY_ID = "select_user_by_id"
Q_UPDATE_PASSWORD = "update_password"
Q_DELETE_USER = "delete_user"
Q_UPSERT_PERMISSION = "upsert_permission"
Q_SELECT_PERMISSIONS = "select_permissions"
Q_DELETE_PERMISSIONS = "delete_permissions"
Q_DELETE_PERMISSION_SUBSYSTEM = "delete_permission_subsystem"
Q_INSERT_BLACKLIST = "insert_blacklist"
Q_SELECT_BLACKLIST = "select_blacklist_jti"
Q_DELETE_EXPIRED_BLACKLIST = "delete_expired_blacklist"
Q_UPSERT_REFRESH_TOKEN = "upsert_refresh_token"
Q_SELECT_REFRESH_TOKEN = "select_refresh_token"
Q_DELETE_REFRESH_TOKEN = "delete_refresh_token"
Q_INSERT_WEBAUTHN = "insert_webauthn"
Q_SELECT_WEBAUTHN_USER = "select_webauthn_user"
Q_SELECT_WEBAUTHN_ID = "select_webauthn_id"
Q_SELECT_WEBAUTHN_COUNTS = "select_webauthn_counts"
Q_DELETE_WEBAUTHN = "delete_webauthn"
Q_UPDATE_WEBAUTHN_SIGN_COUNT = "update_webauthn_sign_count"
Q_SELECT_ALL_USERS = "select_all_users"
Q_SELECT_USERS_WITH_PERMS = "select_users_with_perms"
Q_SELECT_USER_JWT_SECRET = "select_user_jwt_secret"
Q_UPDATE_JWT_SECRET = "update_jwt_secret"
# ---------------------------------------------------------------------------
# Schema DDL
# ---------------------------------------------------------------------------
INIT_SQL = """
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT UNIQUE NOT NULL,
password_hash TEXT NOT NULL,
jwt_secret TEXT NOT NULL,
created_at INTEGER NOT NULL DEFAULT (unixepoch())
);
CREATE TABLE IF NOT EXISTS permissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT NOT NULL REFERENCES users(username) ON DELETE CASCADE,
subsystem TEXT NOT NULL,
level TEXT NOT NULL CHECK (level IN ('read', 'rw')),
UNIQUE(username, subsystem)
);
CREATE TABLE IF NOT EXISTS token_blacklist (
jti TEXT PRIMARY KEY,
token_type TEXT NOT NULL,
expires INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS refresh_tokens (
username TEXT UNIQUE NOT NULL,
jti TEXT NOT NULL,
issued_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS webauthn_creds (
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT NOT NULL REFERENCES users(username) ON DELETE CASCADE,
credential_id TEXT NOT NULL,
public_key TEXT NOT NULL,
sign_count INTEGER NOT NULL DEFAULT 0,
name TEXT NOT NULL DEFAULT '',
transports TEXT NOT NULL DEFAULT '[]',
UNIQUE(username, credential_id)
);
CREATE TABLE IF NOT EXISTS init_sequence (
seq INTEGER PRIMARY KEY
);
"""
class Transaction:
"""Context manager for database transactions.
Provides BEGIN/COMMIT/ROLLBACK semantics. Auto-commit is suppressed
inside the transaction block.
Usage:
with db.in_transaction() as tx:
tx.run(Q_INSERT_USER, ("user1", "hash"))
tx.run(Q_UPSERT_PERMISSION, ("user1", "firewall", "rw"))
"""
def __init__(self, parent: Database) -> None:
self._parent = parent
def __enter__(self) -> Transaction:
self._parent._begin()
self._parent._suppress_auto_commit()
return self
def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> bool:
try:
if exc_type is None:
self._parent._commit()
else:
self._parent._rollback()
finally:
self._parent._restore_auto_commit()
return False
def query(self, query_id: str, params: tuple = ()) -> list[dict]:
return self._parent.query(query_id, params)
def run(self, query_id: str, params: tuple = ()) -> int:
return self._parent.run(query_id, params)
def run_one(self, query_id: str, params: tuple = ()) -> int | dict:
return self._parent.run_one(query_id, params)
class Database(ABC):
"""Abstract database interface.
All subsystems interact with the database through this interface.
Queries are identified by string IDs (e.g. Q_INSERT_USER) never
raw SQL strings.
Connection is cached via the ``conn`` property. Prepared statements
are auto-cached on first use.
"""
QUERY_MAP: ClassVar[dict[str, str]] = {}
def __init__(self, connection_string: str) -> None:
self._connection_string = connection_string
self._prepared: dict[str, Any] = {}
self._in_transaction = False
# Per-thread connections: backend connection objects (e.g. sqlite3)
# are bound to the thread that created them. The Flask WebUI runs
# requests in worker threads while the daemon uses a single event-loop
# thread, so each thread lazily gets its own connection.
self._local = threading.local()
@property
def conn(self) -> Any:
"""Return this thread's cached database connection, creating it lazily."""
conn = getattr(self._local, "conn", None)
if conn is None:
conn = self._connect(self._connection_string)
self._local.conn = conn
return conn
@abstractmethod
def _connect(self, cs: str) -> Any: ...
@abstractmethod
def _prepare(self, sql: str) -> Any: ...
@abstractmethod
def _execute(self, stmt: Any, params: tuple) -> tuple[list[dict], int]:
"""Execute a prepared statement. Returns (rows, rowcount)."""
...
@abstractmethod
def _last_insert_id(self, stmt: Any) -> int | dict: ...
@abstractmethod
def _execute_direct(self, sql: str) -> None:
"""Execute raw SQL without prepared statements (for DDL, not implemented by base)."""
def _begin(self) -> None: # noqa: B027
"""Begin a transaction (not implemented by base)."""
def _commit(self) -> None: # noqa: B027
"""Commit a transaction (not implemented by base)."""
def _rollback(self) -> None: # noqa: B027
"""Rollback a transaction (not implemented by base)."""
def _suppress_auto_commit(self) -> None: # noqa: B027
"""Suppress auto-commit (not implemented by base)."""
def _restore_auto_commit(self) -> None: # noqa: B027
"""Restore auto-commit (not implemented by base)."""
def query(self, query_id: str, params: tuple = ()) -> list[dict]:
"""Execute a SELECT query. Returns list of row dicts."""
stmt = self._get_prepared(query_id)
rows, _ = self._execute(stmt, params)
return rows
def run(self, query_id: str, params: tuple = ()) -> int:
"""Execute an INSERT/UPDATE/DELETE. Returns affected row count."""
stmt = self._get_prepared(query_id)
_, count = self._execute(stmt, params)
return count
def run_one(self, query_id: str, params: tuple = ()) -> int | dict:
"""Execute and return the last insert ID or row dict."""
stmt = self._get_prepared(query_id)
_, _ = self._execute(stmt, params)
return self._last_insert_id(stmt)
def in_transaction(self) -> Transaction:
"""Return a transaction context manager."""
return Transaction(self)
def init_tables(self) -> None:
"""Create schema tables if they don't exist."""
self._execute_direct(INIT_SQL)
def _get_prepared(self, query_id: str) -> Any:
if query_id not in self._prepared:
if query_id not in self.QUERY_MAP:
raise KeyError(f"Unknown query ID: {query_id!r}")
self._prepared[query_id] = self._prepare(self.QUERY_MAP[query_id])
return self._prepared[query_id]
# ---------------------------------------------------------------------------
# Singleton accessor
# ---------------------------------------------------------------------------
_db_instance: Database | None = None
def _get_backend_name() -> str:
return os.environ.get("VACUUM_WALL_DB_BACKEND", "sqlite")
def _get_db_path() -> str:
return os.environ.get("VACUUM_WALL_DB_PATH", str(PROJECT_DIR / "data" / "auth.db"))
def get_db() -> Database:
"""Return the singleton Database instance.
Creates the instance on first call using the backend specified by
``VACUUM_WALL_DB_BACKEND`` env var (default: sqlite).
Call this at application startup to ensure the DB is initialized.
"""
global _db_instance
if _db_instance is None:
backend = _get_backend_name()
if backend == "sqlite":
from lib.db_sqlite import SQLiteBackend
path = _get_db_path()
_db_instance = SQLiteBackend(path)
Path(path).parent.mkdir(parents=True, exist_ok=True)
_db_instance.init_tables()
_seed_builtin_admin(_db_instance)
else:
raise ValueError(f"Unknown database backend: {backend!r}")
return _db_instance
def _seed_builtin_admin(db: Database) -> None:
"""Create the builtin admin user as a last-resort fallback.
Seeding is skipped when:
- ``VACUUM_WALL_SEED_BUILTIN_ADMIN`` is set to ``0`` bootstrap_auth.py
sets this, since bootstrap creates the operator user itself and must
not leave a hardcoded ``admin`` with an unrecoverable random password, or
- the users table already contains users accounts exist, so bootstrap
(or a prior start) has run.
The seed therefore only fires on a completely empty database, i.e.
bootstrap was genuinely skipped and a service starts first.
The builtin admin has full (rw) access to all subsystems and cannot
be deleted or have permissions modified through the normal API.
"""
if os.environ.get("VACUUM_WALL_SEED_BUILTIN_ADMIN", "1") == "0":
return
from lib.auth_users import ALL_SUBSYSTEMS, BUILTIN_ADMIN_USERNAME
from lib.password import hash_password
# Last-resort guard: only seed when no users exist at all.
if db.query(Q_SELECT_ALL_USERS):
return
# Generate a random password — this fallback should only fire if
# bootstrap_auth.py was skipped. Log the password prominently.
random_password = secrets.token_urlsafe(24)
placeholder_hash = hash_password(random_password)
jwt_secret = secrets.token_urlsafe(32)
try:
with db.in_transaction() as tx:
tx.run_one(
Q_INSERT_USER, (BUILTIN_ADMIN_USERNAME, placeholder_hash, jwt_secret)
)
for subsystem in ALL_SUBSYSTEMS:
tx.run(Q_UPSERT_PERMISSION, (BUILTIN_ADMIN_USERNAME, subsystem, "rw"))
except Exception as exc:
rows = db.query(Q_SELECT_USER_BY_NAME, (BUILTIN_ADMIN_USERNAME,))
if not rows:
raise
logger.warning(
"Concurrent builtin admin seed detected (%s); proceeding with existing user",
exc,
)
return
auth_log = Path("/var/log/vacuum-wall/auth.log")
auth_log_written = False
try:
auth_log.write_text(
f"Builtin admin password: {random_password}\n", encoding="utf-8"
)
import os as _os
_os.chmod(str(auth_log), 0o600)
auth_log_written = True
except OSError:
logger.error("Could not write admin password to %s", auth_log)
logger.warning(
"Builtin admin user created. THIS IS A FALLBACK — bootstrap_auth.py "
"should have run during install. Admin password: %s... (%s)",
random_password[:6],
"see /var/log/vacuum-wall/auth.log for the full password"
if auth_log_written
else "full password NOT written — check the error above",
)
def reset_db_for_test() -> None:
"""Reset the singleton — only for tests."""
global _db_instance
if _db_instance is not None:
_db_instance = None
+185
View File
@@ -0,0 +1,185 @@
"""SQLite backend for Vacuum Wall database.
Concrete implementation of the Database abstract class using SQLite3.
Uses Python 3.13+ sqlite3.Statement for prepared statements.
"""
from __future__ import annotations
import contextlib
import logging
import sqlite3
from typing import Any, ClassVar
from lib.db import (
Q_DELETE_EXPIRED_BLACKLIST,
Q_DELETE_PERMISSION_SUBSYSTEM,
Q_DELETE_PERMISSIONS,
Q_DELETE_REFRESH_TOKEN,
Q_DELETE_USER,
Q_DELETE_WEBAUTHN,
Q_INSERT_BLACKLIST,
Q_INSERT_USER,
Q_INSERT_WEBAUTHN,
Q_SELECT_ALL_USERS,
Q_SELECT_BLACKLIST,
Q_SELECT_PERMISSIONS,
Q_SELECT_REFRESH_TOKEN,
Q_SELECT_USER_BY_ID,
Q_SELECT_USER_BY_NAME,
Q_SELECT_USER_JWT_SECRET,
Q_SELECT_USERS_WITH_PERMS,
Q_SELECT_WEBAUTHN_COUNTS,
Q_SELECT_WEBAUTHN_ID,
Q_SELECT_WEBAUTHN_USER,
Q_UPDATE_JWT_SECRET,
Q_UPDATE_PASSWORD,
Q_UPDATE_WEBAUTHN_SIGN_COUNT,
Q_UPSERT_PERMISSION,
Q_UPSERT_REFRESH_TOKEN,
Database,
)
logger = logging.getLogger(__name__)
class SQLiteBackend(Database):
"""SQLite implementation of the Database interface.
Uses ``sqlite3.Connection.execute()`` for statement execution and
``sqlite3.Row`` for row-factory dict access.
"""
def __init__(self, connection_string: str) -> None:
super().__init__(connection_string)
self._last_rowid: int = 0
QUERY_MAP: ClassVar[dict[str, str]] = {
# Schema init is handled by direct execution, not prepared statements
# init_tables is called as _execute_direct(INIT_SQL)
# Users
Q_INSERT_USER: (
"INSERT INTO users (username, password_hash, jwt_secret) VALUES (?, ?, ?)"
),
Q_SELECT_USER_BY_NAME: (
"SELECT id, username, password_hash, jwt_secret, created_at FROM users WHERE username = ?"
),
Q_SELECT_USER_BY_ID: (
"SELECT id, username, password_hash, jwt_secret, created_at FROM users WHERE id = ?"
),
Q_UPDATE_PASSWORD: "UPDATE users SET password_hash = ? WHERE username = ?",
Q_DELETE_USER: "DELETE FROM users WHERE username = ?",
Q_SELECT_ALL_USERS: (
"SELECT id, username, created_at FROM users ORDER BY username"
),
Q_SELECT_USERS_WITH_PERMS: (
"SELECT u.id, u.username, u.created_at, p.subsystem, p.level "
"FROM users u LEFT JOIN permissions p ON u.username = p.username "
"ORDER BY u.username"
),
Q_SELECT_USER_JWT_SECRET: ("SELECT jwt_secret FROM users WHERE username = ?"),
Q_UPDATE_JWT_SECRET: ("UPDATE users SET jwt_secret = ? WHERE username = ?"),
# Permissions
Q_UPSERT_PERMISSION: (
"INSERT INTO permissions (username, subsystem, level) "
"VALUES (?, ?, ?) "
"ON CONFLICT(username, subsystem) DO UPDATE SET level = excluded.level"
),
Q_SELECT_PERMISSIONS: (
"SELECT subsystem, level FROM permissions WHERE username = ?"
),
Q_DELETE_PERMISSIONS: ("DELETE FROM permissions WHERE username = ?"),
Q_DELETE_PERMISSION_SUBSYSTEM: (
"DELETE FROM permissions WHERE username = ? AND subsystem = ?"
),
# Token blacklist
Q_INSERT_BLACKLIST: (
"INSERT OR IGNORE INTO token_blacklist (jti, token_type, expires) VALUES (?, ?, ?)"
),
Q_SELECT_BLACKLIST: "SELECT jti FROM token_blacklist WHERE jti = ?",
Q_DELETE_EXPIRED_BLACKLIST: "DELETE FROM token_blacklist WHERE expires < ?",
# Refresh tokens
Q_UPSERT_REFRESH_TOKEN: (
"INSERT INTO refresh_tokens (username, jti, issued_at) "
"VALUES (?, ?, ?) "
"ON CONFLICT(username) DO UPDATE SET jti = excluded.jti, issued_at = excluded.issued_at"
),
Q_SELECT_REFRESH_TOKEN: "SELECT username, jti, issued_at FROM refresh_tokens WHERE username = ?",
Q_DELETE_REFRESH_TOKEN: "DELETE FROM refresh_tokens WHERE username = ?",
# WebAuthn
Q_INSERT_WEBAUTHN: (
"INSERT INTO webauthn_creds "
"(username, credential_id, public_key, sign_count, name, transports) "
"VALUES (?, ?, ?, ?, ?, ?)"
),
Q_SELECT_WEBAUTHN_USER: (
"SELECT id, credential_id, public_key, sign_count, name, transports "
"FROM webauthn_creds WHERE username = ?"
),
Q_SELECT_WEBAUTHN_ID: (
"SELECT id, username, credential_id, public_key, sign_count, name, transports "
"FROM webauthn_creds WHERE credential_id = ?"
),
Q_SELECT_WEBAUTHN_COUNTS: (
"SELECT username, COUNT(*) as cred_count FROM webauthn_creds "
"GROUP BY username"
),
Q_UPDATE_WEBAUTHN_SIGN_COUNT: (
"UPDATE webauthn_creds SET sign_count = ? WHERE credential_id = ?"
),
Q_DELETE_WEBAUTHN: "DELETE FROM webauthn_creds WHERE credential_id = ?",
}
def _connect(self, cs: str) -> Any:
"""Create a SQLite connection with WAL mode and row factory."""
conn = sqlite3.connect(cs, isolation_level=None)
conn.execute("PRAGMA journal_mode=WAL")
# Multiple threads/processes hold distinct connections (see
# Database.conn); wait up to 5s for writers instead of failing
# immediately with SQLITE_BUSY.
conn.execute("PRAGMA busy_timeout=5000")
conn.execute("PRAGMA foreign_keys=ON")
conn.row_factory = sqlite3.Row
return conn
def _prepare(self, sql: str) -> Any:
"""Store the SQL string for later execution.
SQLite in-memory DB doesn't support the Python 3.13 conn.prepare()
API, so we store the raw SQL and execute via conn.execute().
"""
return sql
def _execute(self, stmt: Any, params: tuple) -> tuple[list[dict], int]:
"""Execute a prepared statement, returning (rows, rowcount)."""
cursor = self.conn.execute(stmt, params)
self._last_rowid = cursor.lastrowid
result = cursor.fetchall()
rows: list[dict] = []
for row in result:
rows.append(dict(row))
return rows, cursor.rowcount
def _last_insert_id(self, stmt: Any) -> int | dict:
"""Return the last insert row ID from the most recent execute."""
return self._last_rowid
def _begin(self) -> None:
self.conn.execute("BEGIN")
def _commit(self) -> None:
self.conn.execute("COMMIT")
def _rollback(self) -> None:
with contextlib.suppress(sqlite3.Error):
self.conn.execute("ROLLBACK")
def _suppress_auto_commit(self) -> None:
self._in_transaction = True
def _restore_auto_commit(self) -> None:
self._in_transaction = False
def _execute_direct(self, sql: str) -> None:
"""Execute raw SQL without prepared statements (for DDL)."""
self.conn.executescript(sql)
+16 -312
View File
@@ -1,18 +1,15 @@
"""Dnsmasq configuration manager for Vacuum Wall SSL proxy firewall.
"""Dnsmasq config persistence for Vacuum Wall.
Generates /etc/dnsmasq.d/vacuum-wall.conf and manages DHCP range,
static leases, and custom DNS records through sudo.
Provides load/save for the declarative JSON config and upstream-management
helpers used by the sync bus and network handler. All mutation and
apply logic lives in daemon/handlers/dnsmasq.py.
"""
import logging
import subprocess
from copy import deepcopy
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from jinja2 import Environment, FileSystemLoader
from lib.common import deep_merge, ensure_dirs, load_json, save_json
logger = logging.getLogger(__name__)
@@ -22,17 +19,7 @@ CONFIG_DIR = PROJECT_DIR / "config" / "dnsmasq"
DATA_DIR = PROJECT_DIR / "data" / "dnsmasq"
CONFIG_PATH = CONFIG_DIR / "config.json"
FRAGMENTS_DIR = DATA_DIR / "fragments"
DNSMASQ_CONF = "/etc/dnsmasq.d/vacuum-wall.conf"
LEASE_FILE = "/var/lib/dnsmasq/dnsmasq.leases"
ENV = Environment(
loader=FileSystemLoader(str(PROJECT_DIR / "system")),
autoescape=False,
lstrip_blocks=True,
trim_blocks=True,
)
# --- defaults ---
DEFAULT_CFG: dict[str, Any] = {
"dhcp": {
"ranges": [],
@@ -50,8 +37,12 @@ DEFAULT_CFG: dict[str, Any] = {
def get_config() -> dict[str, Any]:
"""Load current dnsmasq config from JSON state file."""
ensure_dirs(CONFIG_DIR, DATA_DIR, FRAGMENTS_DIR)
"""Load current dnsmasq config from JSON state file.
Pure read never writes or creates directories. Returns the in-memory
default when the file is missing; directories and the file are
materialized on the first ``save_config``.
"""
raw = load_json(CONFIG_PATH)
if not raw:
return deepcopy(DEFAULT_CFG)
@@ -66,244 +57,7 @@ def save_config(cfg: dict[str, Any]) -> None:
logger.info("dnsmasq config saved")
def apply_config() -> None:
"""Write generated config to disk via sudo tee, then reload dnsmasq."""
cfg = get_config()
conf_text = generate_conf(cfg)
ensure_dirs(CONFIG_DIR, DATA_DIR, FRAGMENTS_DIR)
subprocess.run(["sudo", "mkdir", "-p", "/etc/dnsmasq.d"], check=True)
subprocess.run(
["sudo", "tee", DNSMASQ_CONF, "--"],
input=conf_text,
capture_output=True,
text=True,
check=True,
)
subprocess.run(
["sudo", "systemctl", "reload", "dnsmasq"],
capture_output=True,
text=True,
check=True,
)
logger.info("dnsmasq config written and reloaded")
# ───────── config generation ─────────────────────────────────────────
def generate_conf(cfg: dict[str, Any]) -> str:
"""Render a complete dnsmasq.conf text block from the config dict."""
dhcp_cfg = cfg.get("dhcp", {})
dns_cfg = cfg.get("dns", {})
interfaces = [
r["interface"] for r in dhcp_cfg.get("ranges", []) if "interface" in r
]
# Fallback: use network-managed interface addresses for listen-address
listen_addresses = []
try:
from lib.network import get_config as _get_net_config
net_cfg = _get_net_config()
for _iface, info in net_cfg.get("interfaces", {}).items():
for addr_str in info.get("addresses", []):
if "/" in addr_str:
addr_str = addr_str.split("/")[0]
listen_addresses.append(addr_str)
except Exception:
pass
tmpl = ENV.get_template("dnsmasq.conf")
return tmpl.render(
timestamp=datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
interfaces=interfaces or None,
listen_addresses=listen_addresses if listen_addresses else None,
dhcp=dhcp_cfg,
dns=dns_cfg,
fragments_dir=str(FRAGMENTS_DIR) if FRAGMENTS_DIR.exists() else None,
)
# ───────── dhcp management ───────────────────────────────────────────
def set_dhcp_range(
iface: str,
start: str,
end: str,
lease_time: str = "12h",
gateway: str | None = None,
dns: str | None = None,
) -> None:
"""Add or replace the DHCP range for a given interface."""
cfg = get_config()
ranges = cfg["dhcp"]["ranges"]
found = False
for i, r in enumerate(ranges):
if r.get("interface") == iface:
ranges[i] = {
"interface": iface,
"start": start,
"end": end,
"lease_time": lease_time,
}
if gateway:
ranges[i]["gateway"] = gateway
if dns:
ranges[i]["dns"] = dns
found = True
break
if not found:
entry: dict[str, Any] = {
"interface": iface,
"start": start,
"end": end,
"lease_time": lease_time,
}
if gateway:
entry["gateway"] = gateway
if dns:
entry["dns"] = dns
ranges.append(entry)
save_config(cfg)
logger.info("DHCP range set for interface '%s': %s-%s", iface, start, end)
def remove_dhcp_range(iface: str, start: str, end: str) -> None:
"""Remove a DHCP range by interface + IP range."""
cfg = get_config()
cfg["dhcp"]["ranges"] = [
r
for r in cfg["dhcp"]["ranges"]
if not (
r.get("interface") == iface
and r.get("start") == start
and r.get("end") == end
)
]
save_config(cfg)
logger.info("DHCP range removed for interface '%s': %s-%s", iface, start, end)
def add_static_lease(mac: str, ip: str, hostname: str | None = None) -> None:
"""Add (or update) a static DHCP lease by MAC address."""
cfg = get_config()
leases = cfg["dhcp"]["static_leases"]
for i, lease in enumerate(leases):
if lease["mac"].lower() == mac.lower():
leases[i].update({"mac": mac, "ip": ip})
if hostname is not None:
leases[i]["hostname"] = hostname
save_config(cfg)
logger.info("Static DHCP lease updated: %s -> %s", mac, ip)
return
entry: dict[str, Any] = {"mac": mac, "ip": ip}
if hostname:
entry["hostname"] = hostname
leases.append(entry)
save_config(cfg)
logger.info("Static DHCP lease added: %s -> %s", mac, ip)
def remove_static_lease(mac: str) -> None:
"""Remove a static DHCP lease by MAC address."""
cfg = get_config()
cfg["dhcp"]["static_leases"] = [
lease
for lease in cfg["dhcp"]["static_leases"]
if lease["mac"].lower() != mac.lower()
]
save_config(cfg)
logger.info("Static DHCP lease removed for MAC %s", mac)
# ───────── dns record management ─────────────────────────────────────
def add_dns_record(name: str, address: str, hostname: str | None = None) -> None:
"""Add or update a custom DNS A record."""
cfg = get_config()
records = cfg["dns"]["custom_records"]
for i, r in enumerate(records):
if r["name"] == name:
records[i].update({"name": name, "address": address})
if hostname is not None:
records[i]["hostname"] = hostname
save_config(cfg)
logger.info("DNS record updated: %s -> %s", name, address)
return
entry: dict[str, Any] = {"name": name, "address": address}
if hostname:
entry["hostname"] = hostname
records.append(entry)
save_config(cfg)
logger.info("DNS record added: %s -> %s", name, address)
def remove_dns_record(name: str) -> None:
"""Remove a custom DNS record by name."""
cfg = get_config()
cfg["dns"]["custom_records"] = [
r for r in cfg["dns"]["custom_records"] if r["name"] != name
]
save_config(cfg)
logger.info("DNS record removed: %s", name)
# ───────── lease table ───────────────────────────────────────────────
def _parse_lease_line(line: str) -> dict[str, Any] | None:
"""Parse one line from dnsmasq.leases into a dict."""
line = line.strip()
if not line or line.startswith("#"):
return None
parts = line.split()
if len(parts) < 3:
return None
try:
ts = datetime.fromtimestamp(int(parts[0]), tz=UTC)
except (ValueError, OSError):
ts = None
return {
"expires_at": ts,
"mac": parts[1],
"ip": parts[2],
"hostname": parts[3] if len(parts) > 3 else "",
"interface": parts[4] if len(parts) > 4 else "",
}
def get_lease_table() -> list[dict[str, Any]]:
"""Read and parse the current dnsmasq lease file."""
leases: list[dict[str, Any]] = []
try:
result = subprocess.run(
["sudo", "cat", LEASE_FILE],
capture_output=True,
text=True,
check=True,
)
for entry in map(_parse_lease_line, result.stdout.splitlines()):
if entry is not None:
leases.append(entry)
except subprocess.CalledProcessError:
pass
return leases
# ───────── upstream / domain helpers ─────────────────────────────────
# ───────── upstream helpers ──────────────────────────────────────────
def set_upstreams(servers: list[str]) -> None:
@@ -322,64 +76,14 @@ def set_domain(domain: str | None) -> None:
logger.info("DNS domain set to '%s'", domain)
# ───────── status / info ─────────────────────────────────────────────
def get_status() -> dict[str, Any]:
"""Return service status, config summary, and current lease count."""
cfg = get_config()
try:
proc = subprocess.run(
["sudo", "systemctl", "is-active", "dnsmasq"],
capture_output=True,
text=True,
)
active = proc.stdout.strip() == "active"
except Exception:
active = False
conf_exists = Path(DNSMASQ_CONF).is_file()
if conf_exists:
try:
with open(DNSMASQ_CONF) as f:
conf_on_disk = f.read()
except PermissionError:
conf_on_disk = ""
else:
conf_on_disk = ""
expected = generate_conf(cfg)
leases = get_lease_table()
return {
"service_active": active,
"config_file_exists": conf_exists,
"config_in_sync": conf_on_disk == expected,
"dhcp_ranges": len(cfg["dhcp"]["ranges"]),
"static_leases": len(cfg["dhcp"]["static_leases"]),
"custom_dns_records": len(cfg["dns"]["custom_records"]),
"upstreams": cfg["dns"]["upstreams"],
"domain": cfg["dns"].get("domain"),
"active_leases": len(leases),
"leases": leases,
}
__all__ = [
"add_dns_record",
"add_static_lease",
"apply_config",
"generate_conf",
"CONFIG_DIR",
"CONFIG_PATH",
"DATA_DIR",
"DEFAULT_CFG",
"FRAGMENTS_DIR",
"get_config",
"get_lease_table",
"get_status",
"remove_dhcp_range",
"remove_dns_record",
"remove_static_lease",
"save_config",
"set_dhcp_range",
"set_domain",
"set_upstreams",
]
+281 -30
View File
@@ -6,9 +6,12 @@ All privileged commands are handled by daemon/handlers/firewall.py.
"""
import logging
from collections.abc import Sequence
from copy import deepcopy
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from xml.etree import ElementTree
from lib.common import load_json, save_json
@@ -22,6 +25,22 @@ CONFIG_FILE: Path = CONFIG_DIR / "config.json"
DEFAULT_CONFIG: dict[str, Any] = {"zones": {}}
# Zones firewalld ships by default. They are always present live and are
# never meaningful to flag as "unmanaged (not in config)".
FIREWALLD_BUILTIN_ZONES: frozenset[str] = frozenset(
{
"block",
"dmz",
"drop",
"external",
"home",
"host",
"internal",
"public",
"trusted",
}
)
# ---------------------------------------------------------------------------
# Internal helpers
@@ -66,16 +85,34 @@ def _parse_interfaces(output: str) -> list[str]:
def _parse_zone_output(zone: str, output: str) -> dict[str, Any]:
"""Parse ``firewall-cmd --zone=Z --list-all`` output."""
"""Parse ``firewall-cmd --zone=Z --list-all`` or a zone block
from ``--list-all-zones`` output.
firewalld emits each rich rule on its own tab-indented continuation
line after an (empty) ``rich rules:`` entry; those lines carry no
colon and are collected into the ``rich-rules`` list.
"""
info: dict[str, Any] = {"name": zone}
last_key = ""
for line in output.splitlines():
line = line.strip()
if not line or ":" not in line:
if not line:
continue
if ":" not in line:
# Continuation line (rich rules); ignore anything else.
if last_key == "rich-rules":
info.setdefault("rich-rules", []).append(line)
continue
key, _, value = line.partition(":")
key = key.strip()
value = value.strip()
# --list-all-zones uses "rich rules" (space) while
# --zone=Z --list-all uses "rich-rules" (hyphen); normalize.
if key == "rich rules":
key = "rich-rules"
last_key = key
if not value:
if key in ("masquerade", "ics"):
info[key] = False
@@ -116,6 +153,117 @@ def _parse_zone_output(zone: str, output: str) -> dict[str, Any]:
return info
def _parse_all_zones_output(output: str) -> dict[str, dict[str, Any]]:
"""Parse the combined ``firewall-cmd --list-all-zones`` output.
Returns a dict mapping each zone name to its parsed info dict
(same structure as ``_parse_zone_output``).
"""
zones: dict[str, dict[str, Any]] = {}
current_name: str | None = None
current_lines: list[str] = []
for raw_line in output.splitlines():
if not raw_line.strip():
continue
# Non-indented line starts a new zone block
if raw_line[0].isspace():
if current_name is not None:
current_lines.append(raw_line.strip())
else:
# Finalize previous zone
if current_name is not None and current_lines:
zones[current_name] = _parse_zone_output(
current_name, "\n".join(current_lines)
)
# Extract zone name (discard trailing parenthetical metadata)
name = raw_line.strip().split()[0]
if "(" in name:
name = name[: name.index("(")]
current_name = name
current_lines = []
# Finalize last zone
if current_name is not None and current_lines:
zones[current_name] = _parse_zone_output(current_name, "\n".join(current_lines))
return zones
# ---------------------------------------------------------------------------
# Service catalog descriptions (firewalld service XML definitions)
# ---------------------------------------------------------------------------
# Built-ins first, /etc second, so user service definitions under
# /etc/firewalld/services override built-ins with the same name.
_SERVICE_XML_DIRS: tuple[Path, ...] = (
Path("/usr/lib/firewalld/services"),
Path("/etc/firewalld/services"),
)
_service_descriptions_cache: dict[str, str] | None = None
def _parse_service_xml(path: Path) -> str:
"""Extract the one-line text from a firewalld service XML definition.
Args:
path: Path to a ``<service>`` XML file.
Returns:
The ``<short>`` text, or ``<description>`` when ``<short>`` is
absent; empty string when neither is present or the file cannot be
read or parsed.
"""
try:
root = ElementTree.parse(path).getroot()
except (OSError, ElementTree.ParseError):
logger.warning("Could not read service definition %s", path, exc_info=True)
return ""
text = root.findtext("short") or root.findtext("description") or ""
return text.strip()
def get_service_descriptions(
dirs: Sequence[Path | str] | None = None,
) -> dict[str, str]:
"""Return a mapping of firewalld service names to one-line descriptions.
Parses the ``*.xml`` service definitions found in *dirs*. When *dirs* is
``None`` the standard system locations are used (see
``_SERVICE_XML_DIRS``) and the result is cached for the process lifetime.
When *dirs* is given the result is computed fresh and nothing is cached.
Unreadable or malformed files are skipped.
Args:
dirs: Directories containing service XML files. ``None`` selects the
default system locations.
Returns:
Dict mapping each service name (file stem) to its description text.
"""
global _service_descriptions_cache
if dirs is None and _service_descriptions_cache is not None:
return dict(_service_descriptions_cache)
search_dirs = [Path(d) for d in dirs] if dirs is not None else _SERVICE_XML_DIRS
descriptions: dict[str, str] = {}
for directory in search_dirs:
try:
entries = sorted(directory.glob("*.xml")) if directory.is_dir() else []
except OSError:
logger.warning("Skipping unreadable service directory %s", directory)
continue
for path in entries:
text = _parse_service_xml(path)
if text:
descriptions[path.stem] = text
if dirs is None:
_service_descriptions_cache = descriptions
return descriptions
# ---------------------------------------------------------------------------
# Helpers for parsing forward-port lines
# ---------------------------------------------------------------------------
@@ -180,9 +328,16 @@ def _ensure_config_file() -> None:
def get_config() -> dict[str, Any]:
"""Return the declarative config from ``config/firewall/config.json``."""
_ensure_config_file()
return load_json(CONFIG_FILE)
"""Return the declarative config from ``config/firewall/config.json``.
Pure read never writes. Returns the in-memory default when the file
is missing; the file is materialized on the first ``save_config`` (or
by the system-config import on first start).
"""
raw = load_json(CONFIG_FILE)
if not raw:
return deepcopy(DEFAULT_CONFIG)
return raw
def save_config(cfg: dict[str, Any]) -> None:
@@ -222,6 +377,14 @@ def _compute_pending_changes(
Pure function no subprocess calls. Caller is responsible for providing
live state (typically from the daemon).
The config is the source of truth for zone interfaces: an absent
``interfaces`` key counts as an empty list, so every config zone is
diffed on interfaces. Likewise the target diff is only reported when the
config carries an explicit target that normalizes to something other
than ``default`` an absent key or a ``default``-normalizing value is
unmanaged (apply never re-sets it). Services, masquerade, rich rules and
forward ports are reported for all config zones.
"""
cfg_zones = cfg.get("zones", {})
@@ -230,9 +393,9 @@ def _compute_pending_changes(
for zone_name, zone_cfg in cfg_zones.items():
live_zone = live_zones.get(zone_name, {})
if not zone_cfg.get("interfaces"):
continue
# The config is the source of truth for zone interfaces: an absent
# key counts as an empty list, so every config zone is diffed.
cfg_ifaces = set(zone_cfg.get("interfaces", []))
live_ifaces = set(live_zone.get("interfaces", []))
if cfg_ifaces != live_ifaces:
@@ -257,29 +420,38 @@ def _compute_pending_changes(
}
)
cfg_target = _normalize_target(zone_cfg.get("target", "DEFAULT"))
live_target = live_zone.get("target", "default")
if cfg_target != live_target:
changes.append(
{
"zone": zone_name,
"type": "target",
"config": cfg_target,
"live": live_target,
}
)
# Target is unmanaged when the config omits the key or the value
# normalizes to "default" (firewalld's implicit target, which apply
# never re-sets). Only an explicit ACCEPT/DROP/REJECT is diffed.
if "target" in zone_cfg and _normalize_target(zone_cfg["target"]) != "default":
cfg_target = _normalize_target(zone_cfg["target"])
live_target = live_zone.get("target", "default")
if cfg_target != live_target:
changes.append(
{
"zone": zone_name,
"type": "target",
"config": cfg_target,
"live": live_target,
}
)
cfg_mq = zone_cfg.get("masquerade", False)
live_mq = live_zone.get("masquerade", False)
if cfg_mq != live_mq:
changes.append(
{
"zone": zone_name,
"type": "masquerade",
"config": cfg_mq,
"live": live_mq,
}
)
# public zone masquerade is not reconciled by apply (it is driven by
# the nftables propagation step in daemon/handlers/firewall.py), so
# reporting it as pending here would advertise a change that never
# happens. Skip it to keep the diff consistent with apply.
if zone_name != "public":
cfg_mq = zone_cfg.get("masquerade", False)
live_mq = live_zone.get("masquerade", False)
if cfg_mq != live_mq:
changes.append(
{
"zone": zone_name,
"type": "masquerade",
"config": cfg_mq,
"live": live_mq,
}
)
cfg_rules = {r.get("rule") for r in zone_cfg.get("rich_rules", [])}
live_rules = set(live_zone.get("rich-rules", []))
@@ -312,7 +484,7 @@ def _compute_pending_changes(
)
for zone_name in live_zones:
if zone_name not in cfg_zones:
if zone_name not in cfg_zones and zone_name not in FIREWALLD_BUILTIN_ZONES:
unknown_live[zone_name] = {
"interfaces": live_zones[zone_name].get("interfaces", []),
}
@@ -324,6 +496,49 @@ def _compute_pending_changes(
}
def validate_coverage(fw_cfg: dict[str, Any], net_cfg: dict[str, Any]) -> list[str]:
"""Return network-managed interfaces with no firewall zone coverage.
Pure compares the declarative firewall config against the network
config; no live state. A managed interface is covered when it appears in
some zone's ``interfaces`` list (an absent key counts as empty), or is
explicitly declared in the top-level ``unmanaged`` list. ``lo`` and
``wg*`` interfaces are never guarded (VPN zones are managed by the
WireGuard sync; loopback is normally zoneless).
Args:
fw_cfg: Firewall declarative config (``zones`` plus optional
top-level ``unmanaged`` list).
net_cfg: Network config (``interfaces`` mapping).
Returns:
Sorted list of uncovered interface names; empty when the config is
valid.
"""
managed = [
name
for name in net_cfg.get("interfaces", {})
if name != "lo" and not name.startswith("wg")
]
if not managed:
return []
covered: set[str] = set()
for zone_cfg in fw_cfg.get("zones", {}).values():
if isinstance(zone_cfg, dict):
covered.update(
i for i in zone_cfg.get("interfaces", []) if isinstance(i, str)
)
unmanaged_raw = fw_cfg.get("unmanaged", [])
unmanaged = (
{i for i in unmanaged_raw if isinstance(i, str)}
if isinstance(unmanaged_raw, list)
else set()
)
return sorted(
name for name in managed if name not in covered and name not in unmanaged
)
def config_pending(state: dict[str, Any]) -> dict[str, Any]:
"""Compare declarative config against firewalld live state, return diff.
@@ -335,11 +550,43 @@ def config_pending(state: dict[str, Any]) -> dict[str, Any]:
return _compute_pending_changes(cfg, live_zones)
def fw_change_summary(zone: str, ctype: str, change: dict[str, Any]) -> str:
"""Build a human-readable summary string for a firewall change."""
if ctype == "interfaces":
config_if = change.get("config", [])
live_if = change.get("live", [])
return f"Zone {zone}: interfaces changed (config: {config_if}, live: {live_if})"
if ctype == "services":
config_sv = change.get("config", [])
live_sv = change.get("live", [])
return f"Zone {zone}: services changed (config: {config_sv}, live: {live_sv})"
if ctype == "rich_rules":
cfg_count = change.get("config_count", 0)
live_count = change.get("live_count", 0)
return (
f"Zone {zone}: rich rules differ (config: {cfg_count}, live: {live_count})"
)
if ctype == "forward_ports":
cfg_count = change.get("config_count", 0)
live_count = change.get("live_count", 0)
return f"Zone {zone}: port forwards differ (config: {cfg_count}, live: {live_count})"
if ctype == "masquerade":
cfg_val = change.get("config", False)
live_val = change.get("live", False)
return f"Zone {zone}: masquerade changed (config: {cfg_val}, live: {live_val})"
if ctype == "target":
cfg_val = change.get("config", "default")
live_val = change.get("live", "default")
return f"Zone {zone}: target changed (config: {cfg_val}, live: {live_val})"
return f"Zone {zone}: {ctype} changed"
__all__ = [
"CONFIG_DIR",
"CONFIG_FILE",
"DATA_DIR",
"DEFAULT_CONFIG",
"FIREWALLD_BUILTIN_ZONES",
"RULES_FILE",
"_compute_pending_changes",
"_ensure_config_file",
@@ -347,12 +594,16 @@ __all__ = [
"_normalize_target",
"_now_iso",
"_parse_active_zones",
"_parse_all_zones_output",
"_parse_forward_ports",
"_parse_interfaces",
"_parse_zone_output",
"config_pending",
"fw_change_summary",
"get_config",
"get_service_descriptions",
"load_backup",
"save_backup",
"save_config",
"validate_coverage",
]
+9
View File
@@ -76,6 +76,15 @@ def setup_logging(level: str | None = None) -> None:
"""
def _open(self):
"""Override to open log file with group-write permissions (0664).
Temporarily clears umask to ensure the file is writable by both the
WebUI process (vacuum-wall user) and daemon process (vacuum-walld user)
when they share a group. On existing stale files, chmod's to 0664.
Returns:
Open file object in append mode.
"""
# Ensure group-write on an existing stale file (e.g. left by the
# other process with a stricter umask at creation time).
with contextlib.suppress(OSError):
+64 -5
View File
@@ -8,6 +8,7 @@ import contextlib
import ipaddress
import json
import logging
from copy import deepcopy
from pathlib import Path
from typing import Any
@@ -78,13 +79,16 @@ __all__ = [
def get_config() -> dict[str, Any]:
"""Read network config from config/network/config.json.
Pure read never writes. Returns the in-memory default when the file
is missing; the file is materialized on the first ``save_config``.
Returns:
Dict with ``interfaces`` mapping interface names to config entries.
"""
if not CONFIG_FILE.exists():
CONFIG_DIR.mkdir(parents=True, exist_ok=True)
save_json(CONFIG_FILE, DEFAULT_CONFIG, indent=2)
return load_json(CONFIG_FILE)
raw = load_json(CONFIG_FILE)
if not raw:
return deepcopy(DEFAULT_CONFIG)
return raw
def save_config(cfg: dict[str, Any]) -> None:
@@ -103,30 +107,73 @@ def save_config(cfg: dict[str, Any]) -> None:
def _emit_str(lines: list[str], key: str, py_key: str, d: dict[str, Any]) -> None:
"""Append a ``Key=Value`` line to *lines* if *py_key* has a non-None value in *d*.
Args:
lines: Target list to append rendered line to.
key: INI key name for the output line.
py_key: Python dict key to look up in *d*.
d: Config entry dict to extract value from.
"""
v = d.get(py_key)
if v is not None:
lines.append(f"{key}={v}")
def _emit_int(lines: list[str], key: str, py_key: str, d: dict[str, Any]) -> None:
"""Append a ``Key=Value`` line to *lines* for integer values.
Args:
lines: Target list to append rendered line to.
key: INI key name for the output line.
py_key: Python dict key to look up in *d*.
d: Config entry dict to extract value from.
"""
v = d.get(py_key)
if v is not None:
lines.append(f"{key}={v}")
def _emit_bool(lines: list[str], key: str, py_key: str, d: dict[str, Any]) -> None:
"""Append a ``Key=yes/no`` line to *lines* if *py_key* has a non-None value in *d*.
Args:
lines: Target list to append rendered line to.
key: INI key name for the output line.
py_key: Python dict key to look up in *d*.
d: Config entry dict to extract value from.
"""
v = d.get(py_key)
if v is not None:
lines.append(f"{key}={'yes' if v else 'no'}")
def _emit_bool_opt(lines: list[str], key: str, py_key: str, d: dict[str, Any]) -> None:
"""Identical to :func:`_emit_bool` — kept for API compatibility.
Args:
lines: Target list to append rendered line to.
key: INI key name for the output line.
py_key: Python dict key to look up in *d*.
d: Config entry dict to extract value from.
"""
v = d.get(py_key)
if v is not None:
lines.append(f"{key}={'yes' if v else 'no'}")
def _emit_any(lines: list[str], key: str, py_key: str, d: dict[str, Any]) -> None:
"""Append a ``Key=Value`` line handling both bool and non-bool types.
Boolean values are rendered as ``yes``/``no``; all other types are
stringified directly.
Args:
lines: Target list to append rendered line to.
key: INI key name for the output line.
py_key: Python dict key to look up in *d*.
d: Config entry dict to extract value from.
"""
v = d.get(py_key)
if v is not None:
if isinstance(v, bool):
@@ -144,7 +191,7 @@ def render_network_file(iface_name: str, cfg_entry: dict[str, Any]) -> str:
todo.md (addresses, gateway, dns, routes, link, dhcp_client, etc.).
Returns:
INI content string ready to write as 50-<name>.network file.
INI content string ready to write as 99-<name>.network file.
"""
lines: list[str] = []
d = cfg_entry
@@ -570,6 +617,18 @@ _IS_LOCAL = [
def _is_local_dns(addr: str) -> bool:
"""Return ``True`` if *addr* falls within a local/private IP range.
Checks loopback, RFC 1918 (10/8, 172.16/12, 192.168/16), link-local
(169.254/16), and their IPv6 equivalents (fc00::/7, fe80::/10).
Args:
addr: IP address string to test.
Returns:
``True`` if the address is local/private, ``False`` otherwise.
Invalid addresses are treated as non-local.
"""
try:
ip = ipaddress.ip_address(addr)
for net in _IS_LOCAL:
+238 -159
View File
@@ -1,7 +1,7 @@
"""Nginx server-block generator for Vacuum Wall SSL proxy firewall.
Manages per-domain SSL reverse proxy configurations, certificate
bootstrap, basic-auth htpasswd files, and nginx reload cycles.
Manages per-domain SSL reverse proxy configurations backed by named backends,
certificate bootstrap, basic-auth htpasswd files, and nginx reload cycles.
"""
import logging
@@ -14,7 +14,7 @@ from typing import Any
from jinja2 import Environment, FileSystemLoader
from lib.acme import find_cert_dir
from lib.common import ensure_dirs, load_json, save_json
from lib.common import _hash_password, ensure_dirs, load_json, save_json
logger = logging.getLogger(__name__)
@@ -47,92 +47,167 @@ DEFAULT_SSL: dict[str, Any] = {
"prefer_server_ciphers": False,
}
WEBUI_BACKEND: dict[str, Any] = {
"label": "Vacuum Wall WebUI",
"builtin": True,
"paths": {
"/": {
"backend": {"host": "127.0.0.1", "port": 9090, "proto": "http"},
"is_management": True,
# The WebUI is protected by JWT at the Flask layer; nginx must
# not gate it with auth_basic (the SPA sends Bearer tokens, which
# suppress the browser's automatic Basic credentials). auth=None
# renders `auth_basic off` even if legacy auth was harvested.
"auth": None,
},
"/ws": {
"backend": {"host": "127.0.0.1", "port": 9091, "proto": "http"},
"is_websocket": True,
},
},
}
DEFAULT_CONFIG: dict[str, Any] = {
"backends": {},
"domains": {},
"ssl": {**DEFAULT_SSL},
}
# ---------------------------------------------------------------------------
# Resolution
def _resolve_paths(
domain_cfg: dict[str, Any], backends: dict[str, Any]
) -> dict[str, Any]:
"""Resolve effective paths from backends[domain_cfg['backend']].paths."""
backend_name = domain_cfg.get("backend", "")
if backend_name and backend_name in backends:
return backends[backend_name].get("paths", {})
return {}
def _resolve_auth(
domain_cfg: dict[str, Any], backends: dict[str, Any]
) -> dict[str, Any] | None:
"""Resolve effective auth: domain auth -> backend auth -> None."""
if "auth" in domain_cfg:
return domain_cfg.get("auth")
backend_name = domain_cfg.get("backend", "")
if backend_name and backend_name in backends:
backend_auth = backends[backend_name].get("auth")
if backend_auth is not None:
return backend_auth
return None
# ---------------------------------------------------------------------------
# Migration
def _migrate_config(raw: dict[str, Any]) -> dict[str, Any]:
"""Migrate legacy config to backends model."""
_ensure_webui_backend(raw)
_migrate_mgmt_domains(raw)
return raw
def _ensure_webui_backend(raw: dict[str, Any]) -> None:
"""Create the builtin webui backend if not yet migrated."""
backends = raw.setdefault("backends", {})
webui = backends.get("webui")
if webui and webui.get("_migrated"):
return
backends["webui"] = deepcopy(WEBUI_BACKEND)
backends["webui"]["_migrated"] = True
def _migrate_mgmt_domains(raw: dict[str, Any]) -> None:
"""Migrate legacy management domains to backend references.
Legacy format: management domains had inline paths pointing to
127.0.0.1:9090 (Flask) and 127.0.0.1:9091 (WebSocket).
New format: domains reference the "webui" backend by name.
Detection heuristic: if both "/" path points to 127.0.0.1:9090
(is_management) and "/ws" path points to 127.0.0.1:9091
(is_websocket), the domain is a management domain and gets migrated.
"""
backends = raw.get("backends", {})
if not backends.get("webui", {}).get("_migrated"):
return
domains = raw.setdefault("domains", {})
for _name, dom in list(domains.items()):
if dom.get("backend") == "webui":
continue # Already migrated
if dom.get("application") == "webui":
del dom["application"]
paths = dom.get("paths", {})
root = paths.get("/", {})
ws = paths.get("/ws", {})
root_backend = root.get("backend", {})
ws_backend = ws.get("backend", {})
# Check if root path points to Flask management backend
is_mgmt_root = root.get("is_management") or (
root_backend.get("host") == "127.0.0.1" and root_backend.get("port") == 9090
)
# Check if WS path points to WebSocket management backend
is_mgmt_ws = ws.get("is_websocket") or (
ws_backend.get("host") == "127.0.0.1" and ws_backend.get("port") == 9091
)
# If both match, migrate: set backend reference, remove inline paths/auth
if is_mgmt_root and is_mgmt_ws:
dom["backend"] = "webui"
dom.pop("paths", None)
dom.pop("auth", None)
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def _migrate_config(raw: dict[str, Any]) -> dict[str, Any]:
"""Migrate legacy config formats to the new paths-based model.
Handles two migrations:
1. Legacy ``management`` top-level key -> path entry under its domain.
2. Domain entries without ``paths`` -> wrap ``backend`` inside ``paths["/"]``.
Args:
raw: Config dict as loaded from disk.
Returns:
The migrated config dict.
"""
# Migrate management key
if "management" in raw and raw["management"] is not None:
mgmt = raw["management"]
mgmt_domain = mgmt.get("domain", "")
if mgmt_domain:
domains = raw.setdefault("domains", {})
if mgmt_domain not in domains:
domains[mgmt_domain] = {
"force_ssl": True,
"paths": {},
}
dom = domains[mgmt_domain]
paths = dom.setdefault("paths", {})
if "/" not in paths:
paths["/"] = {
"backend": {
"host": mgmt.get("backend", {}).get("host", "127.0.0.1"),
"port": mgmt.get("backend", {}).get("port", 9090),
"proto": "http",
},
"is_management": True,
}
if mgmt.get("auth"):
paths["/"]["auth"] = mgmt["auth"]
# Add WebSocket path if not present
if "/ws" not in paths:
paths["/ws"] = {
"backend": {"host": "127.0.0.1", "port": 9091, "proto": "http"},
"is_websocket": True,
}
del raw["management"]
# Migrate domain entries without paths
for dom in raw.get("domains", {}).values():
if "paths" not in dom and "backend" in dom:
dom["paths"] = {
"/": {
"backend": dom.pop("backend"),
"headers": dom.pop("headers", {}),
}
}
return raw
def get_config() -> dict[str, Any]:
"""Load the current nginx config, initializing with defaults if needed.
"""Load the current nginx config (pure read, in-memory migration).
Ensures config and sites directories exist, applies migrations for
legacy formats, then returns the config dict.
Never writes or creates directories. Returns the in-memory default when
the file is missing and applies legacy-format migration in memory, so
read paths (state collectors, apply-time checks) stay side-effect free.
The one-shot on-disk migration runs at daemon startup via
``migrate_config_file``.
Returns:
The complete config dict with ``domains`` and ``ssl`` keys.
The complete config dict with ``backends``, ``domains``, and ``ssl`` keys.
"""
ensure_dirs(CONFIG_DIR, SITES_DIR)
raw = load_json(CONFIG_FILE)
if not raw:
raw = deepcopy(DEFAULT_CONFIG)
if "ssl" not in raw:
raw["ssl"] = deepcopy(DEFAULT_SSL)
if "backends" not in raw:
raw["backends"] = {}
return _migrate_config(raw)
def migrate_config_file() -> bool:
"""Persist the one-shot legacy-format migration, if the file needs it.
Runs at daemon startup so ``get_config`` reads stay pure. Rewrites the
on-disk file only when migration actually changes it.
Returns:
True when the on-disk file was rewritten, False otherwise.
"""
raw = load_json(CONFIG_FILE)
if not raw:
return False
pre = deepcopy(raw)
raw = _migrate_config(raw)
save_config(raw)
return raw
if raw != pre:
save_config(raw)
return True
return False
def save_config(cfg: dict[str, Any]) -> None:
@@ -143,20 +218,21 @@ def save_config(cfg: dict[str, Any]) -> None:
def get_domains() -> list[dict[str, Any]]:
"""Return a list of all configured proxy domains flattened by path.
Each path within a domain becomes a separate entry with domain-level
settings repeated.
Each path within a domain becomes a separate entry. Paths are resolved
from the domain's referenced backend.
Returns:
List of dicts with ``domain``, ``path``, ``backend``, ``online``,
``force_ssl``, and path-level flags.
``force_ssl``, ``backend_name``, and path-level flags.
"""
cfg = get_config()
backends = cfg.get("backends", {})
result: list[dict[str, Any]] = []
for name, dom in cfg.get("domains", {}).items():
site = SITES_DIR / f"{name}.conf"
paths = dom.get("paths", {})
if not paths:
if "backend" not in dom:
continue
site = SITES_DIR / f"{name}.conf"
paths = _resolve_paths(dom, backends)
for ppath, pcfg in paths.items():
entry: dict[str, Any] = {
"domain": name,
@@ -164,6 +240,7 @@ def get_domains() -> list[dict[str, Any]]:
"backend": pcfg.get("backend", {}),
"online": site.exists(),
"force_ssl": dom.get("force_ssl", True),
"backend_name": dom["backend"],
}
if pcfg.get("is_management"):
entry["is_management"] = True
@@ -173,6 +250,33 @@ def get_domains() -> list[dict[str, Any]]:
return result
def get_management_domains() -> list[str]:
"""Return domain names that serve the management UI.
Checks both backend-referenced paths (for migrated configs) and
inline paths (for legacy configs pending migration).
Returns:
List of domain name strings.
"""
cfg = get_config()
backends = cfg.get("backends", {})
domains: list[str] = []
for name, dom in cfg.get("domains", {}).items():
# Check inline paths (pre-migration format)
inline_paths = dom.get("paths", {})
if any(p.get("is_management") for p in inline_paths.values()):
domains.append(name)
continue
# Check backend-referenced paths
backend_name = dom.get("backend", "")
if backend_name and backend_name in backends:
paths = backends[backend_name].get("paths", {})
if any(p.get("is_management") for p in paths.values()):
domains.append(name)
return domains
# ------------------------------------------------------------------
# Domain CRUD
# ------------------------------------------------------------------
@@ -180,57 +284,37 @@ def get_domains() -> list[dict[str, Any]]:
def add_domain(
domain: str,
backend_host: str | None = None,
backend_port: int | None = None,
backend_proto: str = "http",
backend_name: str,
cert: str | None = None,
extra_headers: dict[str, str] | None = None,
paths: dict[str, dict[str, Any]] | None = None,
force_ssl: bool = True,
auth: dict[str, Any] | None = None,
) -> None:
"""Add a new proxy domain with the given backend and optional settings.
"""Add a new proxy domain that references an existing backend.
Args:
domain: Domain name to add.
backend_host: Upstream host to proxy to (legacy mode).
backend_port: Upstream port (legacy mode).
backend_proto: Protocol (``http`` or ``https``; legacy mode).
backend_name: Name of the backend to proxy through.
cert: Optional certificate type identifier.
extra_headers: Optional dict of extra headers to forward (legacy mode).
paths: Optional path-to-config map (new mode). Each path entry must
have a ``backend`` key with ``host``, ``port``, and ``proto``.
force_ssl: Whether to enforce HTTPS redirect.
auth: Optional domain-level auth override.
Raises:
ValueError: If the domain is already configured.
ValueError: If the domain is already configured or backend not found.
"""
cfg = get_config()
if domain in cfg["domains"]:
raise ValueError(f"Domain {domain!r} already configured")
if backend_name not in cfg.get("backends", {}):
raise ValueError(f"Backend {backend_name!r} not found")
if paths is not None:
entry: dict[str, Any] = {
"paths": paths,
"force_ssl": True,
}
if cert is not None:
entry["cert"] = cert
else:
if not backend_host or backend_port is None:
raise ValueError("'backend_host' and 'backend_port' are required")
entry = {
"paths": {
"/": {
"backend": {
"host": backend_host,
"port": int(backend_port),
"proto": backend_proto,
},
"headers": extra_headers or {},
}
},
"force_ssl": True,
}
if cert is not None:
entry["cert"] = cert
entry: dict[str, Any] = {
"backend": backend_name,
"force_ssl": force_ssl,
}
if cert is not None:
entry["cert"] = cert
if auth is not None:
entry["auth"] = auth
cfg["domains"][domain] = entry
save_config(cfg)
@@ -249,12 +333,10 @@ def remove_domain(domain: str) -> None:
def update_domain(domain: str, **kwargs: Any) -> None:
"""Update fields of an existing domain entry in-place.
"""Update domain-level fields of an existing domain entry.
Supports both domain-level keys (``force_ssl``, ``cert``, ``auth``,
``paths``) and paths-level shorthand (``backend``, ``headers`` for
the root path). When ``paths`` is provided, it is fully replaced.
When ``backend`` is provided, it updates ``paths["/"]["backend"]``.
Only domain-level keys are accepted: ``backend``, ``cert``, ``force_ssl``,
``auth``. Path changes must be made on the backend.
Args:
domain: Domain name to update.
@@ -262,36 +344,27 @@ def update_domain(domain: str, **kwargs: Any) -> None:
Raises:
KeyError: If the domain is not configured.
ValueError: If a new backend is specified but doesn't exist.
"""
cfg = get_config()
if domain not in cfg["domains"]:
raise KeyError(f"Domain {domain!r} not configured")
entry = cfg["domains"][domain]
# If paths is given, replace entirely
if "paths" in kwargs:
entry["paths"] = kwargs["paths"]
else:
# Legacy: top-level backend/headers -> paths["/"]
paths = entry.setdefault("paths", {})
if "backend" in kwargs:
root = paths.setdefault("/", {"backend": {}, "headers": {}})
root["backend"] = kwargs["backend"]
if "headers" in kwargs:
root = paths.setdefault("/", {"backend": {}, "headers": {}})
root["headers"] = kwargs["headers"]
new_backend = kwargs.get("backend")
if new_backend:
if new_backend not in cfg.get("backends", {}):
raise ValueError(f"Backend {new_backend!r} not found")
entry["backend"] = new_backend
# Remove legacy top-level keys from domain entry
entry.pop("backend", None)
entry.pop("headers", None)
allowed = ("backend", "cert", "force_ssl", "auth")
for key in allowed:
if key in kwargs and key != "backend":
if key == "auth" and kwargs[key] is None:
entry.pop("auth", None)
else:
entry[key] = kwargs[key]
for key, val in kwargs.items():
if key in ("backend", "headers", "paths"):
continue
if isinstance(val, dict) and key in entry:
entry[key].update(val)
else:
entry[key] = val
save_config(cfg)
logger.info("Proxy domain '%s' updated: %s", domain, list(kwargs.keys()))
@@ -301,11 +374,15 @@ def update_domain(domain: str, **kwargs: Any) -> None:
# ------------------------------------------------------------------
def generate_server_conf(domain_cfg: dict[str, Any]) -> str:
def generate_server_conf(
domain_cfg: dict[str, Any], backends: dict[str, Any] | None = None
) -> str:
"""Render the Jinja template for a domain server block.
Args:
domain_cfg: Domain entry dict including the ``domain`` key.
backends: Optional backends dict for path/auth resolution.
When omitted, falls back to reading from domain inline paths.
Returns:
The complete nginx server-block configuration as a string.
@@ -313,7 +390,14 @@ def generate_server_conf(domain_cfg: dict[str, Any]) -> str:
tmpl = ENV.get_template("nginx/server_block.conf")
acme_home_path = PROJECT_DIR / "data" / "acme"
acme_cert_dir = str(find_cert_dir(domain_cfg["domain"], acme_home_path))
paths = domain_cfg.get("paths", {})
if backends is not None:
paths = _resolve_paths(domain_cfg, backends)
domain_auth = _resolve_auth(domain_cfg, backends)
else:
paths = domain_cfg.get("paths", {})
domain_auth = domain_cfg.get("auth")
has_management = any(p.get("is_management") for p in paths.values())
# Resolve custom cert paths for cert=="file"
cert_cfg = domain_cfg.get("cert")
@@ -330,8 +414,9 @@ def generate_server_conf(domain_cfg: dict[str, Any]) -> str:
cert=domain_cfg.get("cert"),
cert_path=cert_path,
cert_key_path=cert_key_path,
domain_auth=domain_cfg.get("auth"),
domain_auth=domain_auth,
has_management=has_management,
static_root=str(PROJECT_DIR / "webui" / "static"),
acme_cert_dir=acme_cert_dir,
certs_dir=str(PROJECT_DIR / "data" / "certs"),
acme_webroot=str(PROJECT_DIR / "data" / "acme" / "www"),
@@ -382,19 +467,20 @@ def write_acme_challenge() -> None:
def write_all_sites() -> None:
"""Regenerate all site configs from the current config state.
Writes server blocks for every configured domain (now unified, including
any management paths), removes orphaned site files, and ensures the ACME
challenge config is present.
Writes server blocks for every configured domain using backend-resolved
paths, removes orphaned site files, and ensures the ACME challenge
config is present.
"""
ensure_dirs(SITES_DIR)
cfg = get_config()
backends = cfg.get("backends", {})
existing = set(SITES_DIR.iterdir()) if SITES_DIR.exists() else set()
written: set[str] = set()
for name, dom in cfg.get("domains", {}).items():
dom_copy = dict(dom, domain=name)
conf = generate_server_conf(dom_copy)
conf = generate_server_conf(dom_copy, backends)
write_site(name, conf)
written.add(f"{name}.conf")
@@ -533,26 +619,19 @@ def write_htpasswd(user: str, password: str) -> None:
os.replace(tmp, HTPASSWD_FILE)
def _hash_password(password: str) -> str:
"""Hash *password* using SHA-256 crypt (``$5$`` format) via passlib.
Args:
password: Plain-text password to hash.
Returns:
The hashed password string suitable for ``.htpasswd`` (e.g. ``$5$rounds=$``).
"""
from passlib.hash import sha256_crypt
return sha256_crypt.hash(password)
__all__ = [
"WEBUI_BACKEND",
"_ensure_webui_backend",
"_migrate_mgmt_domains",
"_resolve_auth",
"_resolve_paths",
"add_domain",
"apply",
"generate_server_conf",
"get_config",
"get_domains",
"get_management_domains",
"migrate_config_file",
"remove_domain",
"save_config",
"test_config",
+83
View File
@@ -0,0 +1,83 @@
"""Password hashing with Argon2id.
Handles user password storage for the auth DB. Uses argon2-cffi (C
implementation of the Argon2id memory-hard KDF). Random 16-byte salt is
generated per hash by the library.
Argon2id is used for auth user passwords ONLY. Nginx basic-auth htpasswd
files continue to use sha256_crypt (passlib) that's a separate concern
with different constraints (htpasswd format is standardized).
"""
from __future__ import annotations
from argon2 import PasswordHasher
from argon2.exceptions import InvalidHashError, VerifyMismatchError
from argon2.low_level import Type
# Argon2id: OWASP recommended parameters
# 64 MiB memory, 3 iterations, 4 parallel threads
_PH = PasswordHasher(
time_cost=3,
memory_cost=65536, # 64 MiB
parallelism=4,
hash_len=32,
salt_len=16,
type=Type.ID,
)
def hash_password(password: str) -> str:
"""Generate an Argon2id hash of *password*.
A random 16-byte salt is generated automatically by argon2.
The resulting hash string starts with ``$argon2id$`` and encodes
the algorithm version, parameters, salt, and hash output.
Args:
password: Plain-text password string.
Returns:
Full Argon2id hash string (e.g. ``$argon2id$v=19$m=65536,t=3,p=4$...``).
Raises:
TypeError: If *password* is not a string or contains NUL bytes.
"""
return _PH.hash(password)
def verify_password(password: str, hash_string: str) -> bool:
"""Verify *password* against an Argon2id *hash_string*.
The hash string must have been produced by :func:`hash_password`.
Args:
password: Plain-text password to verify.
hash_string: Argon2id hash string to compare against.
Returns:
``True`` if the password matches, ``False`` otherwise.
Raises:
TypeError: If *hash_string* is not a valid Argon2id hash.
"""
try:
_PH.verify(hash_string, password)
return True
except (InvalidHashError, VerifyMismatchError, ValueError):
return False
def needs_rehash(hash_string: str) -> bool:
"""Check if *hash_string* needs to be rehashed with updated parameters.
Returns True if the hash was not produced with the current parameters
of the hasher, indicating it should be rehashed on next login.
Args:
hash_string: Argon2id hash string to check.
Returns:
``True`` if the hash parameters should be upgraded.
"""
return _PH.check_needs_rehash(hash_string)
+517
View File
@@ -0,0 +1,517 @@
"""TypedDict schemas for every state collector's return value.
Single source of truth for the state-store data shapes. The Markdown
reference is ``docs/state-model.md``.
"""
from typing import Any, TypedDict
__all__ = [
"AcmeAccount",
"AcmeCert",
"AcmeState",
"CpuLoad",
"DnsmasqDhcpLease",
"DnsmasqState",
"DnsmasqStatus",
"FirewallInterface",
"FirewallState",
"FirewallZone",
"MemoryStats",
"NetworkdInterface",
"NetworkdState",
"NginxDomain",
"NginxState",
"SwapStats",
"SystemState",
"TrafficStats",
"WgClassStatus",
"WgPeer",
"WgState",
"WgStatus",
"WgStatusPeer",
]
# ── Shared notes ───────────────────────────────────────────────
# Every collector return carries a top-level `timestamp` (ISO-8601).
# Subsystems with a declarative config expose pending state as a
# status dict: `status: {"pending_changes": bool}`, except firewall,
# which uses `pending: {config_pending() result}`.
# ── Firewall ──────────────────────────────────────────────────
class FirewallInterface(TypedDict):
"""A network interface as parsed from `ip link` / `ip addr`.
Attributes:
name: Interface name (e.g. "eth0").
mac: MAC address, or ``None`` if unavailable.
state: Link state from `ip link` ("UP", "DOWN", "UNKNOWN", ...).
mtu: MTU value, or ``None`` if unavailable.
ips: IPv4 addresses as "ip/prefix" strings.
ipv6: IPv6 addresses as "ip/prefix" strings.
zone: Assigned firewalld zone name, or ``None``.
"""
name: str
mac: str | None
state: str
mtu: int | None
ips: list[str]
ipv6: list[str]
zone: str | None
class FirewallZone(TypedDict, total=False):
"""A firewalld zone as parsed from `--list-all-zones`.
The zone dict carries the HYPHENATED key "rich-rules" (see
``lib.firewall._parse_all_zones_output``), which TypedDict fields
cannot express. Additional firewalld keys may also appear:
"sources", "ports", "protocols", "forward-ports", "ics",
"icmp-blocks", "module", "rich-rules".
"""
target: str
interfaces: list[str]
services: list[str]
masquerade: bool
# snake_case `rich_rules` exists only at the top-level FirewallState
# (collector re-derivation, lib/state.py); the zone dict itself uses
# the hyphenated "rich-rules" key.
rich_rules: list[str]
class FirewallState(TypedDict):
"""Complete firewalld state (collector: `_collect_firewall`).
Attributes:
config: Contents of config/firewall/config.json.
active_zones: Zone name assigned interfaces.
default_zone: firewalld default zone name ("--get-default-zone");
catch-all zone for interfaces with no explicit assignment.
interfaces: All system interfaces (see FirewallInterface).
available_services: firewalld service catalog ("--get-services").
service_descriptions: Service name to one-line description, parsed
from the firewalld service XML definitions
(``lib.firewall.get_service_descriptions``).
uncovered_interfaces: Network-config interfaces (excluding ``lo``/``wg*``) not in any live zone (advisory coverage warning, always present).
zones: All zones as runtime dicts (see FirewallZone).
rich_rules: Zone name raw firewalld rich-rule strings.
pending: config_pending() result:
``{pending: [...], needs_apply: bool,
unmanaged_zones: {zone: {interfaces: [...]}}}``.
timestamp: ISO-8601 collection time.
"""
config: dict[str, Any]
active_zones: dict[str, list[str]]
default_zone: str
interfaces: list[FirewallInterface]
available_services: list[str]
service_descriptions: dict[str, str]
uncovered_interfaces: list[str]
zones: dict[str, FirewallZone]
rich_rules: dict[str, list[str]]
pending: dict[str, Any]
timestamp: str
# ── Dnsmasq ───────────────────────────────────────────────────
class DnsmasqDhcpLease(TypedDict):
"""A single dnsmasq DHCP lease from the lease file.
Attributes:
expires: ISO-8601 expiry timestamp, or "" if unparseable.
mac: Client MAC address.
ip: Leased IP address.
hostname: Client hostname (may be "").
interface: Interface the lease was granted on (may be "").
"""
expires: str
mac: str
ip: str
hostname: str
interface: str
class PendingChange(TypedDict):
"""One field-level difference between the applied config and the current
saved config (see `lib.common.deep_diff`).
Attributes:
path: Dotted (or indexed) path to the changed field.
action: "added", "removed", or "changed".
old: Value in the last applied config (None when added).
new: Value in the current config (None when removed).
"""
path: str
action: str
old: Any
new: Any
class DnsmasqStatus(TypedDict):
"""Dnsmasq service status snapshot.
Attributes:
service_active: Whether the dnsmasq systemd unit is active.
config_file_exists: Whether the rendered .conf is on disk.
active_leases: Count of currently active leases.
pending_changes: Whether the config is dirty vs the applied state.
pending_diff: Field-level changes since the last apply (empty when
up to date or when no applied snapshot is recorded).
"""
service_active: bool
config_file_exists: bool
active_leases: int
pending_changes: bool
pending_diff: list[PendingChange]
class DnsmasqState(TypedDict):
"""Dnsmasq state (collector: `_collect_dnsmasq`).
Attributes:
config: config/dnsmasq/config.json, deep-merged with defaults.
status: Service/config status (see DnsmasqStatus).
leases: Active DHCP leases.
timestamp: ISO-8601 collection time.
"""
config: dict[str, Any]
status: DnsmasqStatus
leases: list[DnsmasqDhcpLease]
timestamp: str
# ── Nginx ─────────────────────────────────────────────────────
class NginxDomain(TypedDict, total=False):
"""One flattened domain+path entry (see `_resolve_paths`).
Attributes:
domain: Domain name (config key).
path: Path prefix for this entry.
backend: Resolved backend config dict.
online: Whether a site .conf exists on disk.
force_ssl: Redirect-to-HTTPS flag.
backend_name: Backend config key this domain points at.
cert: Certificate reference (e.g. "acme", "selfsigned", ...).
is_management: Management UI path marker.
is_websocket: WebSocket-capable path marker.
"""
domain: str
path: str
backend: dict[str, Any]
online: bool
force_ssl: bool
backend_name: str
cert: str | None
is_management: bool
is_websocket: bool
class NginxState(TypedDict):
"""Nginx state (collector: `_collect_nginx`).
Attributes:
config: config/nginx/config.json.
domains: Flattened domain entries (one per domain+path).
status: ``{"pending_changes": bool, "pending_diff": list[dict]}``.
timestamp: ISO-8601 collection time.
"""
config: dict[str, Any]
domains: list[NginxDomain]
status: dict[str, Any]
timestamp: str
# ── ACME ──────────────────────────────────────────────────────
class AcmeAccount(TypedDict):
"""ACME account status (see `_parse_account_conf`).
Attributes:
registered: Whether an account exists.
email: Registered email ("" when not registered).
ca: Human-readable CA name ("" when not registered).
key_length: Account key size, or ``None``.
"""
registered: bool
email: str
ca: str
key_length: int | None
class AcmeCert(TypedDict, total=False):
"""One certificate entry from `list_certs()` output.
Attributes:
domain: Certificate domain name.
expiry: Expiry date string.
renewed: Last renewal date string.
status: "valid" | "expired" | "active" | ...
days_remaining: Days until expiry.
Additional keys from `lib.acme.list_certs()` output may appear.
"""
domain: str
expiry: str
renewed: str
status: str
days_remaining: int
class AcmeState(TypedDict):
"""ACME state (collector: `_collect_acme`).
Attributes:
certs: Certificate list (empty when collection failed).
email: Registered ACME email.
account: Account status (see AcmeAccount).
status: Collection status; ``error`` is ``None`` on success or
the failure message when cert collection was not possible.
timestamp: ISO-8601 collection time.
"""
certs: list[AcmeCert]
email: str
account: AcmeAccount
status: dict[str, str | None]
timestamp: str
# ── WireGuard ─────────────────────────────────────────────────
# The collector tracks BOTH a legacy single interface (wg0) and one
# wg-<class> interface per access class.
class WgStatusPeer(TypedDict):
"""A runtime peer parsed from `wg show` output.
Attributes:
public_key: Peer public key.
endpoint: Last-seen endpoint, or ``None``.
allowed_ips: Allowed IP/CIDR list.
latest_handshake: Human-readable last handshake time, or ``None``.
transfer_received: Received-bytes string from the transfer line.
transfer_sent: Sent-bytes string from the transfer line.
persistent_keepalive: Keepalive seconds, or ``None``.
"""
public_key: str
endpoint: str | None
allowed_ips: list[str]
latest_handshake: str | None
transfer_received: str
transfer_sent: str
persistent_keepalive: int | None
class WgClassStatus(TypedDict):
"""Runtime status for one wg-<class> interface.
Attributes:
up: Whether the class interface is up.
interface: ``{public_key, listen_port}`` (empty when down).
peers: Runtime peers of this class interface.
"""
up: bool
interface: dict[str, Any]
peers: list[WgStatusPeer]
class WgStatus(TypedDict):
"""Aggregated WireGuard runtime status.
Attributes:
up: True when any managed interface is up.
interface: Legacy single-interface info (public_key, listen_port).
peers: Legacy single-interface runtime peers.
classes: Per-access-class runtime status (keyed by class name).
pending_changes: Whether the config is dirty vs the applied state.
pending_diff: Field-level changes since the last apply (empty when
up to date or when no applied snapshot is recorded).
"""
up: bool
interface: dict[str, Any]
peers: list[WgStatusPeer]
classes: dict[str, WgClassStatus]
pending_changes: bool
pending_diff: list[PendingChange]
class WgPeer(TypedDict, total=False):
"""A config-file peer (``private_key`` stripped).
Attributes:
name: Peer config key.
public_key: Peer public key.
endpoint: Configured endpoint, or ``None``.
allowed_ips: Allowed IP/CIDR list.
persistent_keepalive: Keepalive seconds, or ``None``.
preshared_key: Preshared key, or ``None``.
"""
name: str
public_key: str
endpoint: str | None
allowed_ips: list[str]
persistent_keepalive: int | None
preshared_key: str | None
class WgState(TypedDict):
"""WireGuard state (collector: `_collect_wireguard`).
Attributes:
config: config/wireguard/config.json; ``private_key`` stripped
from the interface AND from every access class.
status: Runtime status (see WgStatus).
peers: Config peers, private keys stripped.
timestamp: ISO-8601 collection time.
"""
config: dict[str, Any]
status: WgStatus
peers: list[WgPeer]
timestamp: str
# ── Networkd ──────────────────────────────────────────────────
# Matches parse_networkctl_status() output exactly (lib/network.py).
# There is a single combined `addresses` list — no separate
# `ipv6_addresses` or `routes` keys.
class NetworkdInterface(TypedDict):
"""One networkd interface as parsed by `parse_networkctl_status`.
Attributes:
addresses: "ip/prefix" entries, IPv4+IPv6 combined.
gateway: Default-route gateway, or ``None``.
dns: Configured DNS server list.
mac: MAC address, or ``None``.
state: OperationalState (e.g. "routable", "degraded", "off").
link: Link type (e.g. "ether", "loopback", ...).
"""
addresses: list[str]
gateway: str | None
dns: list[str]
mac: str | None
state: str
link: str
class NetworkdState(TypedDict):
"""Networkd state (collector: `_collect_networkd`).
Attributes:
config: config/network/config.json.
interfaces: Runtime state keyed by interface name.
status: ``{"pending_changes": bool}``.
timestamp: ISO-8601 collection time.
"""
config: dict[str, Any]
interfaces: dict[str, NetworkdInterface]
status: dict[str, Any]
timestamp: str
# ── System ────────────────────────────────────────────────────
class CpuLoad(TypedDict):
"""CPU load averages (see /proc/loadavg).
Attributes:
load1: 1-minute load average.
load5: 5-minute load average.
load15: 15-minute load average.
"""
load1: float
load5: float
load15: float
class MemoryStats(TypedDict):
"""Memory usage (see /proc/meminfo).
Attributes:
total: Total memory in bytes.
available: Available memory in bytes.
used: Used memory in bytes.
used_pct: Used percentage (0-100), rounded to 0.1.
"""
total: int
available: int
used: int
used_pct: float
class SwapStats(TypedDict):
"""Swap usage (see /proc/meminfo).
Attributes:
total: Total swap in bytes.
used: Used swap in bytes.
used_pct: Used percentage (0-100), rounded to 0.1.
"""
total: int
used: int
used_pct: float
class TrafficStats(TypedDict):
"""Per-interface traffic counters (see /sys/class/net/<iface>/statistics).
Attributes:
rx_bytes: Total bytes received.
tx_bytes: Total bytes transmitted.
rx_packets: Total packets received.
tx_packets: Total packets transmitted.
"""
rx_bytes: int
tx_bytes: int
rx_packets: int
tx_packets: int
class SystemState(TypedDict):
"""System-wide metrics (collector: `_collect_system`).
Attributes:
load: CPU load averages (see CpuLoad).
memory: Memory usage (see MemoryStats).
swap: Swap usage (see SwapStats).
traffic: Per-interface counters keyed by interface name.
timestamp: ISO-8601 collection time.
"""
load: CpuLoad
memory: MemoryStats
swap: SwapStats
traffic: dict[str, TrafficStats]
timestamp: str
+107 -751
View File
@@ -2,40 +2,33 @@
Collects system state at startup and on demand. Handlers read from the
state instead of invoking subprocesses on every request.
The collectors themselves live in ``daemon/collectors/`` (they make
read-only ``sudo`` queries and so do not belong in ``lib/``). Importing
that package registers them here as a side effect; registration must
happen before the first ``populate()``/``poll()`` call.
"""
import contextlib
import logging
import os
from copy import deepcopy
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, ClassVar
from lib.common import load_json, run, run_proc
from lib.firewall import (
_parse_active_zones,
_parse_zone_output,
)
from lib.firewall import (
config_pending as _config_pending,
)
from lib.network import parse_networkctl_status
logger = logging.getLogger(__name__)
PROJECT_DIR = Path(__file__).resolve().parent.parent
_CA_NAME_MAP: dict[str, str] = {
"letsencrypt": "Let's Encrypt",
"zerossl": "ZeroSSL",
}
_DEFAULT_POLL_INTERVALS: dict[str, int] = {
"firewall": 30,
"wireguard": 10,
"dnsmasq": 10,
"networkd": 10,
"system": 1,
# nginx/acme state derives from config files and rendered artifacts on
# disk; poll so drift (manual edits, out-of-band applies) is re-collected.
"nginx": 60,
"acme": 300,
}
@@ -63,6 +56,7 @@ class State:
"acme",
"wireguard",
"networkd",
"system",
]
def __init__(self) -> None:
@@ -119,6 +113,17 @@ class State:
"""
return self._data.get(subsystem)
def get_snapshot(self) -> dict[str, dict[str, Any] | None]:
"""Return all subsystem state dicts.
Used for the initial WS snapshot on connect.
Returns:
Dict mapping every subsystem name to its state data
(``None`` when not populated or the last collection failed).
"""
return {name: self._data.get(name) for name in self.SUBSYSTEMS}
def set(self, subsystem: str, data: dict[str, Any] | None) -> None:
"""Set state data for *subsystem*.
@@ -277,58 +282,72 @@ def _strip_volatile(
for k in pop_keys:
stripped.pop(k, None)
for vpath in volatile:
# Determine if this is a list-of-dicts pattern
list_marker = vpath.index("[]") if "[]" in vpath else -1
if list_marker != -1:
# Split into prefix (before []), item keys (after [])
prefix = vpath[:list_marker].split(".")
item_keys = (
vpath[list_marker + 3 :].split(".")
if list_marker + 3 < len(vpath)
else []
)
parent = stripped
for seg in prefix:
_strip_volatile_path(stripped, vpath)
return stripped
def _strip_volatile_item(item: dict[str, Any], keys: list[str]) -> None:
"""Recursively strip volatile keys from *item*, handling nested ``[]`` markers."""
for i, k in enumerate(keys):
if "[]" in k:
base_key = k.replace("[]", "")
rest = keys[i + 1 :]
target = item.get(base_key, [])
if isinstance(target, list):
for t in target:
if isinstance(t, dict):
_strip_volatile_item(t, rest)
elif isinstance(target, dict):
for v in target.values():
if isinstance(v, dict):
_strip_volatile_item(v, rest)
return
elif i == len(keys) - 1:
item[k] = None
return
else:
if isinstance(item, dict) and k in item:
item = item[k]
else:
return
def _strip_volatile_path(stripped: dict[str, Any], vpath: str) -> None:
"""Strip a single volatile path from *stripped*, supporting nested ``[]`` markers."""
list_marker = vpath.index("[]") if "[]" in vpath else -1
if list_marker == -1:
segments = vpath.split(".")
parent = stripped
for i, seg in enumerate(segments):
if i == len(segments) - 1:
if isinstance(parent, dict) and seg in parent:
parent[seg] = None
else:
if isinstance(parent, dict) and seg in parent:
parent = parent[seg]
else:
break
if isinstance(parent, list):
items = parent
elif isinstance(parent, dict):
logger.debug(
"_strip_volatile: %s resolved to dict, falling back to .values()",
vpath,
)
items = parent.values()
else:
continue
for item in items:
if isinstance(item, dict):
curr = item
for i, ik in enumerate(item_keys):
if i == len(item_keys) - 1:
curr[ik] = None
else:
if isinstance(curr, dict) and ik in curr:
curr = curr[ik]
else:
break
return
return
# Split into prefix and item keys.
prefix = vpath[:list_marker].split(".")
item_keys = (
vpath[list_marker + 3 :].split(".") if list_marker + 3 < len(vpath) else []
)
parent = stripped
for seg in prefix:
if isinstance(parent, dict) and seg in parent:
parent = parent[seg]
else:
# Scalar/dict path
segments = vpath.split(".")
parent = stripped
for i, seg in enumerate(segments):
if i == len(segments) - 1:
if isinstance(parent, dict) and seg in parent:
parent[seg] = None
else:
if isinstance(parent, dict) and seg in parent:
parent = parent[seg]
else:
break
return stripped
return
if isinstance(parent, list):
items = parent
elif isinstance(parent, dict):
items = list(parent.values())
else:
return
for item in items:
if isinstance(item, dict) and item_keys:
_strip_volatile_item(item, item_keys)
def _diff_layers(
@@ -338,26 +357,36 @@ def _diff_layers(
) -> tuple[bool, bool]:
"""Compare *old* and *new* state using two-layer diff.
Strips ``timestamp`` from both before comparing.
The two-layer strategy distinguishes between:
1. Structural changes (config, topology) triggers full client re-fetch
2. Volatile changes (byte counters, timestamps) triggers lightweight tick
If structural data changed, volatile is suppressed (False) because the
structural change already triggers a full re-fetch, making the volatile
signal redundant.
Args:
old: Previous state data, or ``None`` if not yet populated.
new: New state data from collector.
volatile: Frozenset of volatile field paths.
Returns:
``(structural_changed, volatile_changed)``
``True`` means that layer differs between old and new.
If structural data changed, volatile is always ``False``
(the structural change already triggers a full re-fetch, so
the volatile signal is suppressed).
``(structural_changed, volatile_changed)``.
"""
if old is None:
return (True, True)
# Structural diff: compare with volatile/timestamp fields zeroed
# Structural diff: compare with volatile fields zeroed out, plus timestamp
# removed. If these differ, the configuration or topology has changed.
pop_keys = frozenset(("timestamp",))
old_struct = _strip_volatile(old, volatile, pop_keys)
new_struct = _strip_volatile(new, volatile, pop_keys)
structural = old_struct != new_struct
# Volatile diff: compare without timestamp
# Volatile diff: only relevant if structural is unchanged. Compare full
# data (minus timestamp). If this differs, only volatile fields changed
# (e.g. WireGuard transfer counters), and a lightweight tick suffices.
volatile_changed = False
if not structural:
old_no_ts = {k: v for k, v in old.items() if k != "timestamp"}
@@ -367,694 +396,21 @@ def _diff_layers(
return (structural, volatile_changed)
# ---------------------------------------------------------------------------
# Firewall collector
# ---------------------------------------------------------------------------
def _now_iso() -> str:
"""Return the current UTC time as an ISO 8601 string."""
return datetime.now(UTC).isoformat()
def _fp_to_str(fp: dict[str, Any]) -> str:
"""Convert a port-forward dict to a compact string representation.
Args:
fp: Port-forward entry containing port and proto keys.
Returns:
Comma-separated string of key=value pairs (e.g. ``port=443,proto=tcp``).
"""
parts = [f"port={fp['port']}", f"proto={fp['proto']}"]
if "toaddr" in fp:
parts.append(f"toaddr={fp['toaddr']}")
if "toport" in fp:
parts.append(f"toport={fp['toport']}")
return "/".join(parts)
def _collect_firewall() -> dict[str, Any]:
"""Return the complete current state of firewalld.
Returns:
Dict containing firewall zones, interfaces, rules, config, and
pending changes.
"""
zone_names = run(["firewall-cmd", "--get-zones"], sudo=True).split()
active_raw = run(["firewall-cmd", "--get-active-zones"], sudo=True)
active = _parse_active_zones(active_raw)
services = run(["firewall-cmd", "--get-services"], sudo=True).split() or []
link_out = run(["ip", "-o", "link", "show"], sudo=True)
addr_out = run(["ip", "-o", "addr", "show"], sudo=True)
iface_map: dict[str, dict[str, Any]] = {}
for line in link_out.splitlines():
if not line:
continue
parts = line.split()
if len(parts) < 2:
continue
raw_name = parts[1].rstrip(":").split("@")[0]
iface_state = "UNKNOWN"
mtu = None
mac = None
for i, p in enumerate(parts):
if p == "state" and i + 1 < len(parts):
iface_state = parts[i + 1]
if p == "mtu" and i + 1 < len(parts):
mtu = int(parts[i + 1])
if p.startswith("link/ether") and i + 1 < len(parts):
mac = parts[i + 1]
iface_map[raw_name] = {
"name": raw_name,
"mac": mac,
"state": iface_state,
"mtu": mtu,
"ips": [],
"ipv6": [],
"zone": None,
}
for line in addr_out.splitlines():
if not line:
continue
parts = line.split()
if len(parts) < 4:
continue
addr_name = parts[1].split("@")[0]
addr_key = "ipv6" if parts[2] == "inet6" else "ips"
for entry in iface_map.values():
if entry["name"] == addr_name:
entry[addr_key].append(parts[3])
break
for zone_name, ifaces in active.items():
for raw_if in ifaces:
for entry in iface_map.values():
if entry["name"] == raw_if:
entry["zone"] = zone_name
break
ifaces = list(iface_map.values())
zones: dict[str, dict[str, Any]] = {}
for zn in zone_names:
try:
zones[zn] = _parse_zone_output(
zn, run(["firewall-cmd", f"--zone={zn}", "--list-all"], sudo=True)
)
except Exception:
continue
# Load config
fw_config_path = PROJECT_DIR / "config" / "firewall" / "config.json"
config_data = {}
if fw_config_path.exists():
with contextlib.suppress(Exception):
config_data = load_json(fw_config_path)
# Pending changes
full_state = {
"active_zones": active,
"interfaces": ifaces,
"available_services": services,
"zones": zones,
"rich_rules": {n: z.get("rich-rules", []) for n, z in zones.items()},
"timestamp": _now_iso(),
}
pending = {}
with contextlib.suppress(Exception):
pending = _config_pending(full_state)
return {
"active_zones": active,
"interfaces": ifaces,
"available_services": services,
"zones": zones,
"rich_rules": {n: z.get("rich-rules", []) for n, z in zones.items()},
"config": config_data,
"pending": pending,
"timestamp": _now_iso(),
}
register_collector("firewall", _collect_firewall)
register_volatile(
"firewall",
frozenset(
{
"interfaces[].ips",
"interfaces[].ipv6",
}
),
)
# ---------------------------------------------------------------------------
# DNSMasq collector
# ---------------------------------------------------------------------------
def _collect_dnsmasq() -> dict[str, Any]:
"""Collect dnsmasq status, config, and leases.
Returns:
Dict containing config, service status, leases, and timestamp.
"""
CONFIG_DIR = PROJECT_DIR / "config" / "dnsmasq"
CONFIG_PATH = CONFIG_DIR / "config.json"
DNSMASQ_CONF = "/etc/dnsmasq.d/vacuum-wall.conf"
LEASE_FILE = "/var/lib/dnsmasq/dnsmasq.leases"
DEFAULT_CFG: dict[str, Any] = {
"dhcp": {"ranges": [], "static_leases": []},
"dns": {
"upstreams": ["8.8.8.8", "1.1.1.1"],
"domain": None,
"custom_records": [],
},
}
# Load config
cfg: dict[str, Any] = {}
if CONFIG_PATH.exists():
try:
raw = load_json(CONFIG_PATH)
if raw:
from lib.common import deep_merge
cfg = deep_merge(deepcopy(DEFAULT_CFG), raw)
else:
cfg = deepcopy(DEFAULT_CFG)
except Exception:
cfg = deepcopy(DEFAULT_CFG)
else:
cfg = deepcopy(DEFAULT_CFG)
# Service status
service_active = False
try:
proc = run_proc(["systemctl", "is-active", "dnsmasq"], sudo=True)
service_active = proc.stdout.strip() == "active"
except Exception:
pass
# Leases
leases: list[dict[str, Any]] = []
try:
result = run_proc(["cat", LEASE_FILE], sudo=True, check=True)
for line in result.stdout.splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
parts = line.split()
if len(parts) < 3:
continue
try:
ts = datetime.fromtimestamp(int(parts[0]), tz=UTC)
except (ValueError, OSError):
ts = None
leases.append(
{
"expires_at": ts,
"mac": parts[1],
"ip": parts[2],
"hostname": parts[3] if len(parts) > 3 else "",
"interface": parts[4] if len(parts) > 4 else "",
}
)
except RuntimeError:
pass
# Check config file on disk
conf_exists = Path(DNSMASQ_CONF).is_file()
return {
"config": cfg,
"status": {
"service_active": service_active,
"config_file_exists": conf_exists,
"active_leases": len(leases),
},
"leases": leases,
"timestamp": _now_iso(),
}
register_collector("dnsmasq", _collect_dnsmasq)
# dnsmasq has no volatile fields — leases change slowly enough to treat as structural
# ---------------------------------------------------------------------------
# Nginx collector
# ---------------------------------------------------------------------------
def _collect_nginx() -> dict[str, Any]:
"""Collect nginx config and domains list.
Returns:
Dict containing config, domains, and timestamp.
"""
CONFIG_DIR = PROJECT_DIR / "config" / "nginx"
CONFIG_FILE = CONFIG_DIR / "config.json"
SITES_DIR = PROJECT_DIR / "data" / "nginx" / "sites-enabled"
DEFAULT_SSL: dict[str, Any] = {
"protocols": "TLSv1.2 TLSv1.3",
"ciphers": (
"ECDHE-ECDSA-AES128-GCM-SHA256:"
"ECDHE-RSA-AES128-GCM-SHA256:"
"ECDHE-ECDSA-AES256-GCM-SHA384:"
"ECDHE-RSA-AES256-GCM-SHA384:"
"ECDHE-ECDSA-CHACHA20-POLY1305:"
"ECDHE-RSA-CHACHA20-POLY1305"
),
"prefer_server_ciphers": False,
}
default_cfg: dict[str, Any] = {
"domains": {},
"ssl": deepcopy(DEFAULT_SSL),
}
cfg = deepcopy(default_cfg)
if CONFIG_FILE.exists():
try:
raw = load_json(CONFIG_FILE)
if raw:
from lib.common import deep_merge
cfg = deep_merge(default_cfg, raw)
if "ssl" not in cfg:
cfg["ssl"] = deepcopy(DEFAULT_SSL)
except Exception:
pass
# Build flattened domains list (one entry per path)
domains: list[dict[str, Any]] = []
for name, dom in cfg.get("domains", {}).items():
site = SITES_DIR / f"{name}.conf"
paths = dom.get("paths", {})
if not paths:
continue
for ppath, pcfg in paths.items():
entry: dict[str, Any] = {
"domain": name,
"path": ppath,
"backend": pcfg.get("backend", {}),
"online": site.exists() if SITES_DIR.exists() else False,
"force_ssl": dom.get("force_ssl", True),
"cert": dom.get("cert"),
}
if pcfg.get("is_management"):
entry["is_management"] = True
if pcfg.get("is_websocket"):
entry["is_websocket"] = True
domains.append(entry)
return {
"config": cfg,
"domains": domains,
"timestamp": _now_iso(),
}
register_collector("nginx", _collect_nginx)
# ---------------------------------------------------------------------------
# ACME collector
# ---------------------------------------------------------------------------
def _resolve_ca_name(ca_server: str) -> str:
"""Map a CA server identifier to its human-readable name.
Uses prefix matching sorted by longest prefix first to avoid
shorter prefixes winning (e.g. "letsencrypt" matching before
"letsencrypt.org").
Args:
ca_server: Raw CA server string from acme.sh config.
Returns:
Human-readable name, or unchanged string if no match.
"""
for prefix, name in sorted(
_CA_NAME_MAP.items(), key=lambda x: len(x[0]), reverse=True
):
if ca_server.startswith(prefix):
return name
return ca_server
def _parse_account_conf(acme_home: Path | None = None) -> dict[str, Any]:
"""Parse acme.sh account information and return account status dict.
Checks three sources in order:
1. Legacy ``.account.conf`` file (acme.sh v2.x format)
2. Declarative ``config/acme/config.json`` (saved by the registration
handler with ``email`` and ``ca`` fields)
Args:
acme_home: Optional override for ACME home directory. Falls back
to ``ACME_HOME`` env var or ``PROJECT_DIR/data/acme``.
Returns:
Dict with ``registered``, ``email``, ``ca``, and
``key_length`` keys. If no account is found, ``registered`` is
``False`` with empty / ``None`` values.
"""
if acme_home is None:
acme_home_env = os.environ.get("ACME_HOME", str(PROJECT_DIR / "data" / "acme"))
acme_home = Path(acme_home_env)
default = {
"registered": False,
"email": "",
"ca": "",
"key_length": None,
}
# 1. Legacy .account.conf (acme.sh v2.x)
account_path = acme_home / ".account.conf"
if account_path.is_file():
try:
text = account_path.read_text()
except OSError:
pass
else:
email = ""
ca_raw = ""
key_length = None
for line in text.splitlines():
if line.startswith("ACME_LEEMAIL="):
email = line.split("=", 1)[1].strip().strip("'\"")
elif line.startswith("ACME_MCA="):
ca_raw = line.split("=", 1)[1].strip().strip("'\"")
elif line.startswith("ACME_CERTKEYSIZE="):
raw_val = line.split("=", 1)[1].strip().strip("'\"")
key_length = int(raw_val) if raw_val.isdigit() else None
if email and ca_raw:
return {
"registered": True,
"email": email,
"ca": _resolve_ca_name(ca_raw),
"key_length": key_length,
}
# 2. Declarative config (saved by register_account / set_email handlers)
# Modern acme.sh (v3.x) stores account data in per-CA JSON files
# (ca/<server>/account.json) — we can't reliably parse those without
# walking the directory, so fall back to the declarative config
# which the handlers keep in sync.
# Derive project root from acme_home (acme_home is at <root>/data/acme).
try:
project_root = acme_home.parent.parent # data/acme → data → project root
acme_cfg = project_root / "config" / "acme" / "config.json"
data = load_json(acme_cfg)
email = (data.get("email") or "").strip()
ca_raw = (data.get("ca") or "").strip()
if email and ca_raw:
return {
"registered": True,
"email": email,
"ca": _resolve_ca_name(ca_raw),
"key_length": None,
}
except (OSError, ValueError):
pass
return default
def _get_acme_email() -> str:
"""Read the ACME ``acme.sh`` email from the account config file.
Falls back to the declarative ACME config (config/acme/config.json)
if acme.sh account has not been registered yet.
"""
from lib.acme import _read_acme_email
return _read_acme_email()
def _collect_acme() -> dict[str, Any]:
"""Collect ACME certificate list and email.
Returns:
Dict containing certificate details and registered email.
"""
email = _get_acme_email()
certs: list[dict[str, Any]] = []
try:
from lib.acme import (
_days_until,
_has_auto_renew,
_parse_list_output,
_run_acme,
)
raw = _run_acme(["--list"])
entries = _parse_list_output(raw)
acme_home_env = os.environ.get("ACME_HOME", str(PROJECT_DIR / "data" / "acme"))
acme_home = Path(acme_home_env)
for entry in entries:
main = entry.get("main_domain", "")
if not main:
continue
san_domains = [
d.strip()
for d in entry.get("san_domains", "").split(",")
if d.strip() and d.strip().lower() != "no"
]
cert_dir = acme_home / main
days = _days_until(entry.get("renew", ""))
certs.append(
{
"domain": main,
"issuer": entry.get("ca", ""),
"expiry": entry.get("renew", ""),
"days_remaining": days,
"expired": days is not None and days <= 0,
"cert_path": str(cert_dir / "fullchain.cer"),
"key_path": str(cert_dir / f"{main}.key"),
"ca_path": str(cert_dir / "ca.cer"),
"issued_at": entry.get("created", ""),
"expires_at": entry.get("renew", ""),
"days_until_expiry": days,
"auto_renew": _has_auto_renew(main),
"san_domains": san_domains,
}
)
except Exception:
logger.warning(
"ACME state collection failed, returning empty cert list",
exc_info=True,
)
raise
account = _parse_account_conf()
return {
"certs": certs,
"email": email,
"account": account,
"timestamp": _now_iso(),
}
register_collector("acme", _collect_acme)
# ---------------------------------------------------------------------------
# WireGuard collector
# ---------------------------------------------------------------------------
def _collect_wireguard() -> dict[str, Any]:
"""Collect WireGuard config, status, and peers.
Returns:
Dict containing interface config, runtime status, and peers.
"""
CONFIG_PATH = PROJECT_DIR / "config" / "wireguard" / "config.json"
DEFAULT_CONFIG: dict[str, Any] = {
"interface": {
"name": "wg0",
"listen_port": 51820,
"private_key": "",
"public_key": "",
"addresses": ["10.137.0.1/24"],
"post_up": None,
"post_down": None,
},
"peers": {},
}
from lib.common import deep_merge
cfg: dict[str, Any] = deepcopy(DEFAULT_CONFIG)
if CONFIG_PATH.exists():
try:
raw = load_json(CONFIG_PATH)
if raw:
cfg = deep_merge(deepcopy(DEFAULT_CONFIG), raw)
except Exception:
pass
# Safe config (strip private key)
safe = dict(cfg)
if "interface" in safe:
safe["interface"] = dict(safe["interface"])
safe["interface"].pop("private_key", None)
# Peers list (safe)
peers: list[dict[str, Any]] = []
for name, info in cfg.get("peers", {}).items():
entry = dict(info)
entry["name"] = name
entry.pop("private_key", None)
peers.append(entry)
# Runtime status
status: dict[str, Any] = {"up": False, "interface": {}, "peers": []}
name = cfg["interface"]["name"]
peer_name = name if isinstance(name, str) else "wg0"
try:
res = run_proc(["wg", "show", peer_name], sudo=True, check=False)
if res.returncode == 0:
raw = res.stdout.strip()
current_peer: dict[str, Any] | None = None
status_peers: list[dict[str, Any]] = []
for line in raw.splitlines():
line = line.strip()
if not line:
continue
if line.startswith("interface:"):
status["up"] = True
status["interface"] = {}
current_peer = None
continue
if line.startswith("public key:"):
status["interface"]["public_key"] = line.split(":", 1)[1].strip()
continue
if line.startswith("listening port:"):
status["interface"]["listen_port"] = int(
line.split(":", 1)[1].strip()
)
continue
if line.startswith("fwmark:"):
status["interface"]["fwmark"] = line.split(":", 1)[1].strip()
continue
if line.startswith("peer:"):
cur_key = line.split(":", 1)[1].strip()
current_peer = {
"public_key": cur_key,
"endpoint": None,
"allowed_ips": [],
"latest_handshake": None,
"transfer_received": "0",
"transfer_sent": "0",
"persistent_keepalive": None,
}
status_peers.append(current_peer)
continue
if current_peer is None:
continue
if line.startswith("endpoint:"):
current_peer["endpoint"] = line.split(":", 1)[1].strip()
elif line.startswith("allowed ips:"):
current_peer["allowed_ips"] = (
line.split(":", 1)[1].strip().split(", ")
)
elif line.startswith("latest handshake:"):
current_peer["latest_handshake"] = line.split(":", 1)[1].strip()
elif line.startswith("transfer:"):
rest = line.split(":", 1)[1].strip().split(", ")
if rest:
current_peer["transfer_received"] = rest[0].strip()
if len(rest) > 1:
current_peer["transfer_sent"] = rest[1].strip()
elif line.startswith("persistent-keepalive:"):
with contextlib.suppress(ValueError):
current_peer["persistent_keepalive"] = int(
line.split(":", 1)[1].strip()
)
status["peers"] = status_peers
except Exception:
pass
return {
"config": safe,
"status": status,
"peers": peers,
"timestamp": _now_iso(),
}
register_collector("wireguard", _collect_wireguard)
register_volatile(
"wireguard",
frozenset(
{
"status.peers[].transfer_received",
"status.peers[].transfer_sent",
"status.peers[].latest_handshake",
}
),
)
# ---------------------------------------------------------------------------
# Networkd collector
# ---------------------------------------------------------------------------
def _collect_networkd() -> dict[str, Any]:
"""Collect networkd interface state from networkctl.
Returns:
Dict with interface runtime state parsed from networkctl output.
Returns empty data if networkctl is not available.
"""
result: dict[str, dict[str, Any]] = {}
try:
raw = run(["networkctl", "status", "--json=short", "--all"], sudo=True)
result = parse_networkctl_status(raw)
if not result:
return {"interfaces": {}, "timestamp": _now_iso()}
except Exception:
return {
"interfaces": {},
"timestamp": _now_iso(),
}
return {
"interfaces": result,
"timestamp": _now_iso(),
}
register_collector("networkd", _collect_networkd)
register_volatile(
"networkd",
frozenset(
{
"interfaces[].addresses",
}
),
)
__all__ = [
"PROJECT_DIR",
"_COLLECTORS",
"_DEFAULT_POLL_INTERVALS",
"_VOLATILE",
"State",
"_diff_layers",
"_now_iso",
"_strip_volatile",
"register_collector",
"register_volatile",
"state",
]
+1046
View File
File diff suppressed because it is too large Load Diff
+991
View File
@@ -0,0 +1,991 @@
"""System config import on daemon startup.
On first start, vacuum-walld parses each subsystem's live system config
file and writes the canonical JSON config. This reconciles any drift
caused by install.sh or manual edits to system files.
"""
import contextlib
import logging
import re
from copy import deepcopy
from pathlib import Path
from typing import Any
from lib.common import (
_APPLY_HASH_KEY,
_LAST_APPLIED_CONFIG_KEY,
load_json,
run,
save_json,
stamp_applied,
)
from lib.firewall import _live_target_to_config, _parse_all_zones_output
logger = logging.getLogger(__name__)
PROJECT_DIR = Path(__file__).resolve().parent.parent
def _safe_int(val: str) -> int | str:
"""Try to parse *val* as int; return original string on failure."""
with contextlib.suppress(ValueError):
return int(val)
return val
DNSMASQ_CONF = Path("/etc/dnsmasq.d/vacuum-wall.conf")
WG_CONF = Path("/etc/wireguard/wg0.conf")
NETWORKD_DIR = Path("/etc/systemd/network")
NGINX_SITES_DIR = PROJECT_DIR / "data" / "nginx" / "sites-enabled"
DNSTART = "# ---- vacuum-wall managed dnsmasq configuration ----"
DNEND = "# ---- end vacuum-wall config ----"
def import_all() -> list[str]:
"""Run all subsystem imports. Returns list of subsystems that were updated."""
updated: list[str] = []
for fn, name in [
(import_dnsmasq, "dnsmasq"),
(import_firewall, "firewall"),
(import_wireguard, "wireguard"),
(import_networkd, "network"),
(import_nginx, "nginx"),
]:
try:
if fn():
updated.append(name)
except Exception:
logger.warning("Import failed for %s", name, exc_info=True)
return updated
def _carry_apply_meta(cfg: dict[str, Any], existing: dict[str, Any]) -> None:
"""Preserve apply bookkeeping when adopting live system state.
Imported content replaces the declarative config but must not destroy
the applied-state baseline. When *existing* carries apply meta keys,
they are copied over so pending-change detection and cancel-all keep
working against the last-applied baseline. When no baseline exists
(first import), *cfg* is stamped as applied the imported content is
exactly the state the system is currently running.
"""
if _APPLY_HASH_KEY in existing or _LAST_APPLIED_CONFIG_KEY in existing:
for key in (_APPLY_HASH_KEY, _LAST_APPLIED_CONFIG_KEY):
if key in existing:
cfg[key] = deepcopy(existing[key])
else:
stamp_applied(cfg)
# ------------------------------------------------------------------
# Dnsmasq
# ------------------------------------------------------------------
def import_dnsmasq() -> bool:
"""Parse /etc/dnsmasq.d/vacuum-wall.conf -> config/dnsmasq/config.json."""
if not DNSMASQ_CONF.exists():
logger.debug("Skipping dnsmasq: %s not found", DNSMASQ_CONF)
return False
try:
text = DNSMASQ_CONF.read_text()
except OSError:
logger.debug("Skipping dnsmasq: cannot read %s", DNSMASQ_CONF)
return False
# Extract managed block
start_idx = text.find(DNSTART)
end_idx = text.rfind(DNEND)
if start_idx == -1 or end_idx == -1 or end_idx <= start_idx:
logger.debug("Skipping dnsmasq: no managed markers found")
return False
block = text[start_idx + len(DNSTART) : end_idx]
try:
cfg = _parse_dnsmasq_block(block)
except Exception:
logger.warning("Failed to parse dnsmasq config", exc_info=True)
return False
cfg_path = PROJECT_DIR / "config" / "dnsmasq" / "config.json"
existing: dict[str, Any] = {}
if cfg_path.exists():
existing = load_json(cfg_path)
if _cfgs_equal(existing, cfg):
logger.debug("Skipping dnsmasq: config already matches")
return False
_carry_apply_meta(cfg, existing)
save_json(cfg_path, cfg)
summary = f"upstreams={len(cfg.get('dns', {}).get('upstreams', []))}, ranges={len(cfg.get('dhcp', {}).get('ranges', []))}"
logger.info("Imported dnsmasq config from %s: %s", DNSMASQ_CONF, summary)
return True
def _parse_dnsmasq_block(block: str) -> dict[str, Any]:
"""Parse a dnsmasq managed block into JSON config dict."""
upstreams: list[str] = []
domains: str | None = None
ranges: list[dict[str, Any]] = []
static_leases: list[dict[str, Any]] = []
custom_records: list[dict[str, Any]] = []
for line in block.splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
if line.startswith("server="):
upstreams.append(line.split("=", 1)[1].strip())
elif line == "no-resolv":
upstreams = []
elif line.startswith("domain="):
domains = line.split("=", 1)[1].strip()
elif line.startswith("dhcp-range="):
rng = _parse_dhcp_range(line.split("=", 1)[1])
ranges.append(rng)
elif line.startswith("dhcp-option="):
target = _attach_dhcp_option(line.split("=", 1)[1], ranges)
if target:
_set_dhcp_option_option(target, line.split("=", 1)[1])
elif line.startswith("dhcp-host="):
lease = _parse_dhcp_host(line.split("=", 1)[1])
if lease:
static_leases.append(lease)
elif line.startswith("addr/"):
rec = _parse_addr_directive(line)
if rec:
custom_records.append(rec)
elif line.startswith("host-record="):
pass # paired with addr/, skip
return {
"dhcp": {
"ranges": ranges,
"static_leases": static_leases,
},
"dns": {
"upstreams": upstreams,
"domain": domains,
"custom_records": custom_records,
},
}
def _parse_dhcp_range(value: str) -> dict[str, Any]:
"""Parse dhcp-range value into dict."""
parts = value.split(",")
if parts[0].startswith("set:"):
iface = parts[0][4:]
addr_parts = parts[1:4]
else:
iface = None
addr_parts = parts[0:3]
rng: dict[str, Any] = {}
if iface:
rng["interface"] = iface
rng["start"] = addr_parts[0] if len(addr_parts) > 0 else ""
rng["end"] = addr_parts[1] if len(addr_parts) > 1 else ""
rng["lease_time"] = addr_parts[2] if len(addr_parts) > 2 else "12h"
return rng
def _attach_dhcp_option(
value: str, ranges: list[dict[str, Any]]
) -> dict[str, Any] | None:
"""Parse dhcp-option and return the target range to attach to."""
m = re.match(r"tag:([^,]+),(\d+),(.+)", value)
if not m:
return None
tag_iface = m.group(1)
for rng in reversed(ranges):
if rng.get("interface") == tag_iface:
return rng
return None
def _set_dhcp_option_option(target: dict[str, Any], value: str) -> None:
"""Parse dhcp-option value string and set gateway/dns on target range."""
m = re.match(r"tag:[^,]+,(\d+),(.+)", value)
if not m:
return
option_code = m.group(1)
option_val = m.group(2).strip()
if option_code == "3":
target["gateway"] = option_val
elif option_code == "6":
target["dns"] = option_val
def _parse_dhcp_host(value: str) -> dict[str, Any] | None:
"""Parse dhcp-host value into dict."""
parts = value.split(",")
if len(parts) < 2:
return None
lease: dict[str, Any] = {
"mac": parts[0].strip().lower(),
"ip": parts[1].strip(),
}
if len(parts) >= 3:
lease["hostname"] = parts[2].strip()
return lease
def _parse_addr_directive(line: str) -> dict[str, Any] | None:
"""Parse addr/name/addr into dict."""
parts = line.split("/")
if len(parts) < 3:
return None
return {"name": parts[1].strip(), "address": parts[2].strip()}
# ------------------------------------------------------------------
# WireGuard
# ------------------------------------------------------------------
def import_wireguard() -> bool:
"""Parse /etc/wireguard/wg0.conf -> config/wireguard/config.json."""
try:
exists = WG_CONF.exists()
except OSError:
# Parent dir may be unreadable to the daemon user (e.g. /etc/wireguard
# is 0700). Treat as not present rather than failing the import.
logger.debug("Skipping wireguard: cannot stat %s", WG_CONF)
return False
if not exists:
logger.debug("Skipping wireguard: %s not found", WG_CONF)
return False
try:
text = WG_CONF.read_text()
except OSError:
logger.debug("Skipping wireguard: cannot read %s", WG_CONF)
return False
try:
cfg = _parse_wireguard_conf(text)
except Exception:
logger.warning("Failed to parse wireguard config", exc_info=True)
return False
cfg_path = PROJECT_DIR / "config" / "wireguard" / "config.json"
existing: dict[str, Any] = {}
if cfg_path.exists():
existing = load_json(cfg_path)
if _cfgs_equal(existing, cfg):
logger.debug("Skipping wireguard: config already matches")
return False
_carry_apply_meta(cfg, existing)
save_json(cfg_path, cfg)
peer_count = len(cfg.get("peers", {}))
logger.info("Imported wireguard config from %s: peers=%d", WG_CONF, peer_count)
return True
def _parse_wireguard_conf(text: str) -> dict[str, Any]:
"""Parse wg-quick INI format into JSON config dict.
Uses a simple state machine: [Interface] section populates the interface
dict; each [Peer] section accumulates into current_peer until the next
section header triggers _flush_peer() to commit it.
"""
interface: dict[str, Any] = {
"name": "wg0",
"listen_port": 51820,
"private_key": "",
"public_key": "",
"addresses": ["10.137.0.1/24"],
"post_up": None,
"post_down": None,
}
peers: dict[str, dict[str, Any]] = {}
current_section = None
current_peer_name = None
current_peer: dict[str, Any] | None = None
def _flush_peer() -> None:
"""Flush the current peer dict into the peers map if it has a public key.
Resets ``current_peer`` and ``current_peer_name`` to ``None``,
preparing for the next [Peer] section.
Note:
Only peers with a ``public_key`` are stored; sections without
a key (malformed or incomplete) are silently skipped.
"""
nonlocal current_peer, current_peer_name
if (
current_peer is not None
and current_peer_name
and current_peer.get("public_key")
):
peers[current_peer_name] = current_peer
current_peer = None
current_peer_name = None
for line in text.splitlines():
stripped = line.strip()
# Section headers — allow trailing content like comments
m = re.match(r"^\[(\w+)\](.*)?$", stripped)
if m:
section_name = m.group(1)
if section_name == "Interface":
current_section = "interface"
_flush_peer()
elif section_name == "Peer":
current_section = "peer"
_flush_peer()
# Look for name in comment after [Peer]
m2 = re.match(r"^\[Peer\]\s*#\s*(.+)", stripped)
current_peer_name = m2.group(1).strip() if m2 else None
current_peer = {}
else:
current_section = None
_flush_peer()
continue
if not stripped or stripped.startswith("#"):
continue
if "=" not in stripped:
continue
key, _, val = stripped.partition("=")
key = key.strip()
val = val.strip()
if current_section == "interface":
if key == "PrivateKey":
interface["private_key"] = val
elif key == "Address":
interface["addresses"] = [a.strip() for a in val.split(",")]
elif key == "ListenPort":
parsed = _safe_int(val)
if isinstance(parsed, int):
interface["listen_port"] = parsed
elif key == "PostUp":
interface["post_up"] = val
elif key == "PostDown":
interface["post_down"] = val
elif current_section == "peer" and current_peer is not None:
if key == "PublicKey":
current_peer["public_key"] = val
if current_peer_name is None:
current_peer_name = val
elif key == "PresharedKey":
current_peer["preshared_key"] = val
elif key == "Endpoint":
current_peer["endpoint"] = val
elif key == "AllowedIPs":
current_peer["allowed_ips"] = [a.strip() for a in val.split(",")]
elif key == "PersistentKeepalive":
parsed = _safe_int(val)
current_peer["persistent_keepalive"] = (
parsed if isinstance(parsed, int) else None
)
# Flush any remaining peer
if (
current_peer is not None
and current_peer_name
and current_peer.get("public_key")
):
peers[current_peer_name] = current_peer
return {
"interface": interface,
"peers": peers,
}
# ------------------------------------------------------------------
# Networkd
# ------------------------------------------------------------------
def import_networkd() -> bool:
"""Parse /etc/systemd/network/*.network -> config/network/config.json."""
if not NETWORKD_DIR.exists():
logger.debug("Skipping network: %s not found", NETWORKD_DIR)
return False
network_files = sorted(NETWORKD_DIR.glob("*.network"))
if not network_files:
logger.debug("Skipping network: no *.network files")
return False
parsed_interfaces: dict[str, dict[str, Any]] = {}
for nf in network_files:
# Extract interface name from filename, stripping optional priority prefix:
# 99-eth0.network -> eth0
# eth0.network -> eth0
base = nf.stem
if "-" in base and base.split("-", 1)[0].isdigit():
iface_name = base.split("-", 1)[1]
else:
iface_name = base
try:
parsed_interface = _parse_network_file(nf)
if parsed_interface:
parsed_interfaces[iface_name] = parsed_interface
except Exception:
logger.warning("Failed to parse %s", nf, exc_info=True)
if not parsed_interfaces:
logger.debug("Skipping network: no valid .network files")
return False
cfg_path = PROJECT_DIR / "config" / "network" / "config.json"
existing: dict[str, Any] = load_json(cfg_path, {"interfaces": {}})
interfaces_cfg = existing.setdefault("interfaces", {})
# Only add/update interfaces with .network files; don't remove interfaces
# without a file (they may be pending apply).
changed = False
for name, entry in parsed_interfaces.items():
if name not in interfaces_cfg or not _cfgs_equal(interfaces_cfg[name], entry):
interfaces_cfg[name] = entry
changed = True
if not changed:
logger.debug("Skipping network: config already matches")
return False
save_json(cfg_path, existing)
logger.info(
"Imported network config from %s/*.network: interfaces=%s",
NETWORKD_DIR,
", ".join(parsed_interfaces.keys()),
)
return True
def _parse_network_file(path: Path) -> dict[str, Any] | None:
"""Parse a .network INI file into interface config dict.
State machine: [Match] section is skipped; [Link] keys go to iface["link"];
[Network] keys go directly on iface. Numbered sections ([Address#N], [Route#N])
accumulate into cur_addr / cur_route dicts until a section boundary triggers
_flush() to commit them into the corresponding list.
"""
text = path.read_text()
iface: dict[str, Any] = {}
cur_section: str | None = None
cur_addr: dict[str, Any] | None = None
cur_route: dict[str, Any] | None = None
def _flush() -> None:
"""Commit accumulated address/route dicts into the iface lists."""
nonlocal cur_addr, cur_route
if cur_addr is not None:
if "address" in cur_addr and len(cur_addr) == 1:
iface.setdefault("addresses", []).append(cur_addr["address"])
elif cur_addr:
iface.setdefault("addresses", []).append(cur_addr)
cur_addr = None
if cur_route is not None and cur_route:
iface.setdefault("routes", []).append(cur_route)
cur_route = None
for line in text.splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#"):
continue
m = re.match(r"^\[(.+)\]$", stripped)
if m:
_flush()
sec = m.group(1)
if sec.startswith("Address"):
cur_section = sec
cur_addr = {}
elif sec.startswith("Route"):
cur_section = sec
cur_route = {}
else:
cur_section = sec
continue
if "=" not in stripped:
continue
key, _, val = stripped.partition("=")
key = key.strip()
val = val.strip()
if cur_addr is not None:
cur_addr = _parse_addr_key(cur_addr, key, val)
continue
if cur_route is not None:
cur_route = _parse_route_key(cur_route, key, val)
continue
_parse_network_section(iface, cur_section, key, val)
_flush()
return iface if iface else None
def _parse_addr_key(entry: dict[str, Any], key: str, val: str) -> dict[str, Any]:
"""Parse an [Address] key-value pair, return updated entry."""
if key == "Address":
entry["address"] = val
elif key == "Label":
entry["label"] = val
elif key == "Scope":
entry["scope"] = val
elif key == "RouteMetric":
parsed = _safe_int(val)
if isinstance(parsed, int):
entry["route_metric"] = parsed
elif key == "DuplicateAddressDetection":
entry["duplicate_address_detection"] = val
elif key == "ManageTemporaryAddress":
entry["manage_temporary_address"] = _parse_bool(val)
elif key == "AddPrefixRoute":
entry["add_prefix_route"] = _parse_bool(val)
return entry
def _parse_route_key(entry: dict[str, Any], key: str, val: str) -> dict[str, Any]:
"""Parse a [Route] key-value pair, return updated entry."""
if key == "Destination":
entry["destination"] = val
elif key == "Gateway":
entry["gateway"] = val
elif key == "Metric":
parsed = _safe_int(val)
if isinstance(parsed, int):
entry["metric"] = parsed
elif key == "Table":
entry["table"] = val
elif key == "Type":
entry["type"] = val
elif key == "Scope":
entry["scope"] = val
elif key == "GatewayOnLink":
entry["gateway_on_link"] = _parse_bool(val)
elif key == "IPv6Preference":
entry["ipv6_preference"] = val
elif key == "MTUBytes":
parsed = _safe_int(val)
if isinstance(parsed, int):
entry["mtu_bytes"] = parsed
return entry
def _parse_network_section(
iface: dict[str, Any], section: str | None, key: str, val: str
) -> None:
"""Parse a [Match]/[Link]/[Network] key-value pair into iface dict."""
if section == "Match":
return
if section == "Link":
link = iface.setdefault("link", {})
_set_link_key(link, key, val)
return
if section == "Network":
_set_network_key(iface, key, val)
return
def _set_link_key(link: dict[str, Any], key: str, val: str) -> None:
"""Parse a [Link] section key-value pair and set the corresponding config field.
Maps systemd-networkd Link INI keys to snake_case config keys.
Boolean keys (ARP, Multicast, etc.) are auto-converted via ``_parse_bool``.
Args:
link: Link config dict to populate.
key: INI key name from the .network file.
val: Value string from the .network file.
"""
if key == "MTUBytes":
parsed = _safe_int(val)
if isinstance(parsed, int):
link["mtu_bytes"] = parsed
elif key == "MACAddress":
link["mac_address"] = val
elif key in ("ARP", "Multicast", "AllMulticast", "Promiscuous", "Unmanaged"):
pk = {
"ARP": "arp",
"Multicast": "multicast",
"AllMulticast": "all_multicast",
"Promiscuous": "promiscuous",
"Unmanaged": "unmanaged",
}[key]
link[pk] = _parse_bool(val)
elif key == "ActivationPolicy":
link["activation_policy"] = val
elif key == "RequiredForOnline":
link["required_for_online"] = val
def _set_network_key(iface: dict[str, Any], key: str, val: str) -> None:
"""Parse a [Network] section key-value pair and set the corresponding config field.
Maps systemd-networkd Network INI keys to snake_case config dict keys.
Comma-separated values (DNS, Domains, etc.) are split into lists.
Boolean and integer keys are auto-converted.
Args:
iface: Interface config dict to populate.
key: INI key name from the .network file.
val: Value string from the .network file.
"""
if key == "DHCP":
iface["dhcp"] = val
elif key == "Gateway":
iface["gateway"] = val
elif key == "IPv6Gateway":
iface["ipv6_gateway"] = val
elif key == "DNS":
for d in val.split(","):
d = d.strip()
if d:
iface.setdefault("dns", []).append(d)
elif key == "IPv6DNS":
for d in val.split(","):
d = d.strip()
if d:
iface.setdefault("ipv6_dns", []).append(d)
elif key == "Domains":
for d in val.split(","):
d = d.strip()
if d:
iface.setdefault("domains", []).append(d)
elif key == "IPv6Domains":
for d in val.split(","):
d = d.strip()
if d:
iface.setdefault("ipv6_domains", []).append(d)
elif key == "DNSDefaultRoute":
iface["dns_default_route"] = _parse_bool(val)
elif key == "BindCarrier":
iface.setdefault("bind_carrier", []).append(val)
elif key == "IgnoreCarrierLoss":
iface["ignore_carrier_loss"] = val
elif key == "KeepConfiguration":
iface["keep_configuration"] = val
elif key == "ConfigureWithoutCarrier":
iface["configure_without_carrier"] = _parse_bool(val)
elif key == "LinkLocalAddressing":
iface["link_local_addressing"] = val
elif key == "IPv6LinkLocalAddressGenerationMode":
iface["ipv6_link_local_address_generation_mode"] = val
elif key == "IPv6StableSecretAddress":
iface["ipv6_stable_secret_address"] = val
elif key == "IPv4LLStartAddress":
iface["ipv4_ll_start_address"] = val
elif key == "IPv4LLRoute":
iface["ipv4_ll_route"] = _parse_bool(val)
elif key == "DefaultRouteOnDevice":
iface["default_route_on_device"] = _parse_bool(val)
elif key == "IPv6HopLimit":
parsed = _safe_int(val)
if isinstance(parsed, int):
iface["ipv6_hop_limit"] = parsed
elif key == "IPv6RetransmissionTimeSec":
iface["ipv6_retransmission_time_sec"] = val
elif key == "IPv4DuplicateAddressDetectionTimeoutSec":
iface["ipv4_duplicate_address_detection_timeout_sec"] = val
elif key == "IPv4ReversePathFilter":
iface["ipv4_reverse_path_filter"] = val
elif key == "IPv4AcceptLocal":
iface["ipv4_accept_local"] = _parse_bool(val)
elif key == "IPv4RouteLocalnet":
iface["ipv4_route_localnet"] = _parse_bool(val)
elif key == "IPv4ProxyARP":
iface["ipv4_proxy_arp"] = _parse_bool(val)
elif key == "IPv4ProxyARPPrivateVLAN":
iface["ipv4_proxy_arp_private_vlan"] = _parse_bool(val)
elif key == "IPv6ProxyNDP":
iface["ipv6_proxy_ndp"] = _parse_bool(val)
elif key == "IPv6ProxyNDPAddress":
iface["ipv6_proxy_ndp_address"] = val
elif key == "IPv6SendRA":
iface["ipv6_send_ra"] = _parse_bool(val)
elif key == "MPLSRouting":
iface["m_pls_routing"] = _parse_bool(val)
elif key == "KeepMaster":
iface["keep_master"] = _parse_bool(val)
elif key == "IPFamily":
iface["ip_family"] = val
def _parse_bool(val: str) -> bool | str:
"""Parse common boolean representations to bool, return as-is otherwise."""
if val.lower() in ("yes", "true", "1", "on"):
return True
if val.lower() in ("no", "false", "0", "off"):
return False
return val
# ------------------------------------------------------------------
# Nginx
# ------------------------------------------------------------------
def import_nginx() -> bool:
"""Parse data/nginx/sites-enabled/*.conf -> config/nginx/config.json.
Bootstraps config.json on hosts that already have rendered sites
(repo reinstalled over an existing data/ dir). If the declarative
config already exists it wins: sites are vacuum-wall's own generated
output ("do not edit manually") and re-parsing them is lossy backend
references get flattened to inline paths, which render empty nginx
sites and hide domains from the WebUI.
"""
cfg_path = PROJECT_DIR / "config" / "nginx" / "config.json"
if cfg_path.exists():
logger.debug("Skipping nginx: %s already exists", cfg_path)
return False
if not NGINX_SITES_DIR.exists():
logger.debug("Skipping nginx: %s not found", NGINX_SITES_DIR)
return False
conf_files = sorted(NGINX_SITES_DIR.glob("*.conf"))
sites: dict[str, dict[str, Any]] = {}
for cf in conf_files:
# Skip acme challenge
if cf.name == "_acme-challenge.conf":
continue
domain = cf.stem
if not domain:
continue
try:
parsed = _parse_nginx_site(cf, domain)
if parsed:
sites[domain] = parsed
except Exception:
logger.warning("Failed to parse nginx site %s", cf, exc_info=True)
if not sites:
logger.debug("Skipping nginx: no valid site files")
return False
existing: dict[str, Any] = load_json(cfg_path, {"domains": {}, "ssl": {}})
domains_cfg = existing.setdefault("domains", {})
changed = False
for name, entry in sites.items():
if name not in domains_cfg or not _cfgs_equal(domains_cfg[name], entry):
domains_cfg[name] = entry
changed = True
if not changed:
logger.debug("Skipping nginx: config already matches")
return False
save_json(cfg_path, existing)
logger.info(
"Imported nginx config from %s/*.conf: domains=%s",
NGINX_SITES_DIR,
", ".join(sites.keys()),
)
return True
def _parse_nginx_site(path: Path, domain: str) -> dict[str, Any] | None:
"""Parse a vacuum-wall nginx site conf into domain config dict."""
text = path.read_text()
# Check it's our file
if "# Auto-generated by Vacuum Wall" not in text:
logger.debug("Skipping nginx site %s: not vacuum-wall generated", path)
return None
# Determine force_ssl: look for port 80 redirect block
force_ssl = bool(re.search(r"listen\s+80\b", text))
# Parse domain-level auth
domain_auth = None
auth_match = re.search(
r"auth_basic\s+.*;\s*\n\s*auth_basic_user_file\s+(.+?);", text
)
if auth_match:
domain_auth = {"htpasswd": auth_match.group(1).strip()}
# Infer cert type
cert = _infer_cert_type(text, domain)
# Parse location blocks
paths: dict[str, dict[str, Any]] = {}
_parse_location_blocks(text, paths, domain_auth)
entry: dict[str, Any] = {
"force_ssl": force_ssl,
"paths": paths,
}
if cert:
entry["cert"] = cert
if domain_auth:
entry["auth"] = domain_auth
return entry
def _infer_cert_type(text: str, domain: str) -> str | None:
"""Infer cert type from ssl_certificate path."""
m = re.search(r"ssl_certificate\s+(.+?);", text)
if not m:
return None
cert_path = m.group(1).strip()
if "acme" in cert_path:
return "acme"
elif "data/certs" in cert_path:
return "selfsigned"
else:
return "file"
def _parse_location_blocks(
text: str, paths: dict[str, dict[str, Any]], domain_auth: dict[str, Any] | None
) -> None:
"""Extract location blocks and their annotations."""
# Match annotation comment before location block
# # /path -> host:port (WebSocket)
# # /path -> host:port
loc_re = re.compile(
r"#\s*(/[\S]*)\s*->\s*(\S+?)(?:\s*\(WebSocket\))?\s*\n"
r"\s*location\s+\1\s*\{",
re.MULTILINE,
)
for m in loc_re.finditer(text):
ppath = m.group(1)
backend_str = m.group(2)
is_ws = "WebSocket" in m.group(0)
host, _, port_str = backend_str.partition(":")
try:
port = int(port_str)
except ValueError:
port = 80
is_websocket = is_ws
# Check for auth_basic off inside this location block
# Find the location block for this path
block_start = m.end()
# Find matching closing brace
depth = 1
idx = block_start
while idx < len(text) and depth > 0:
if text[idx] == "{":
depth += 1
elif text[idx] == "}":
depth -= 1
idx += 1
block_text = text[block_start:idx]
entry: dict[str, Any] = {
"backend": {
"host": host,
"port": port,
"proto": "http",
},
}
if is_websocket:
entry["is_websocket"] = True
elif "auth_basic off" in block_text:
entry["auth"] = None
if "is_management" in block_text or (
"/api" not in ppath and "management" in block_text.lower()
):
entry["is_management"] = True
# Check for WebSocket path
if ppath == "/ws":
entry["is_websocket"] = True
paths[ppath] = entry
# ------------------------------------------------------------------
# Firewall
# ------------------------------------------------------------------
def import_firewall() -> bool:
"""Run firewall-cmd --list-all-zones -> config/firewall/config.json."""
cfg_path = PROJECT_DIR / "config" / "firewall" / "config.json"
if cfg_path.exists():
logger.debug("Skipping firewall: config already exists at %s", cfg_path)
return False
try:
output = run(["firewall-cmd", "--list-all-zones"], sudo=True)
except RuntimeError:
logger.warning("Import failed for firewall: firewall-cmd unavailable")
return False
try:
zones = _parse_all_zones_output(output)
except Exception:
logger.warning("Failed to parse firewall zones", exc_info=True)
return False
zone_configs: dict[str, dict[str, Any]] = {}
for zone_name, parsed in zones.items():
if not parsed["interfaces"]:
continue
zone_cfg: dict[str, Any] = {
"interfaces": parsed["interfaces"],
"services": parsed["services"],
"masquerade": parsed["masquerade"],
"rich_rules": [{"rule": r} for r in parsed["rich-rules"]],
"forward_ports": parsed["forward-ports"],
}
# Omit the target key when the live target normalizes to firewalld's
# implicit "default" so key-absence is the one canonical "unmanaged"
# notation; keep explicit ACCEPT/DROP/REJECT targets.
target = _live_target_to_config(parsed["target"])
if target != "DEFAULT":
zone_cfg["target"] = target
zone_configs[zone_name] = zone_cfg
if not zone_configs:
logger.debug("Skipping firewall: no zones with interfaces")
return False
# Only reached when the config file is absent: the imported zones are
# exactly what firewalld is running, so stamp them as the applied state.
save_json(cfg_path, stamp_applied({"zones": zone_configs}))
logger.info(
"Imported firewall config: zones=%s",
", ".join(zone_configs.keys()),
)
return True
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
def _cfgs_equal(a: dict[str, Any], b: dict[str, Any]) -> bool:
"""Compare two configs ignoring apply bookkeeping keys."""
from lib.common import strip_apply_meta
return strip_apply_meta(a) == strip_apply_meta(b)
+409
View File
@@ -0,0 +1,409 @@
"""WebAuthn passkey support for Vacuum Wall.
Uses the Duo Labs webauthn library (v3) to handle the FIDO2/WebAuthn ceremony:
registration, authentication, and credential management.
Credential data is stored in the database webauthn_creds table.
Configuration comes from config/auth/config.json (webauthn section).
"""
from __future__ import annotations
import base64
import json
import logging
from typing import Any
from webauthn import (
generate_authentication_options,
generate_registration_options,
options_to_json,
verify_authentication_response,
verify_registration_response,
)
from webauthn.helpers.cose import COSEAlgorithmIdentifier
from webauthn.helpers.structs import (
AttestationConveyancePreference,
AuthenticatorSelectionCriteria,
PublicKeyCredentialDescriptor,
ResidentKeyRequirement,
UserVerificationRequirement,
)
from lib.auth import AUTH_CONFIG_PATH
from lib.common import load_json
from lib.db import (
Q_DELETE_WEBAUTHN,
Q_INSERT_WEBAUTHN,
Q_SELECT_WEBAUTHN_COUNTS,
Q_SELECT_WEBAUTHN_ID,
Q_SELECT_WEBAUTHN_USER,
Q_UPDATE_WEBAUTHN_SIGN_COUNT,
get_db,
)
logger = logging.getLogger(__name__)
# Crypto algorithms we support
_SUPPORTED_ALGS = [
COSEAlgorithmIdentifier.ECDSA_SHA_256,
COSEAlgorithmIdentifier.RSASSA_PKCS1_v1_5_SHA_256,
COSEAlgorithmIdentifier.RSASSA_PKCS1_v1_5_SHA_384,
COSEAlgorithmIdentifier.ECDSA_SHA_512,
COSEAlgorithmIdentifier.RSASSA_PKCS1_v1_5_SHA_512,
COSEAlgorithmIdentifier.EDDSA,
]
# b64url helpers
def b64u_encode(b: bytes) -> str:
return base64.urlsafe_b64encode(b).decode("ascii").rstrip("=")
def b64u_decode(s: str) -> bytes:
padding = 4 - len(s) % 4
if padding != 4:
s += "=" * padding
return base64.urlsafe_b64decode(s)
def _get_webauthn_config() -> dict[str, Any]:
"""Load WebAuthn configuration from auth config."""
raw = load_json(AUTH_CONFIG_PATH)
return raw.get("webauthn", {})
def is_enabled() -> bool:
"""Return whether WebAuthn is enabled in config."""
cfg = _get_webauthn_config()
return cfg.get("enabled", True)
def get_rp_name() -> str:
"""Return the Relying Party name from config."""
return _get_webauthn_config().get("rp_name", "Vacuum Wall")
def get_management_domains() -> list[str]:
"""Return domain names eligible for WebAuthn.
Delegates to lib.nginx.get_management_domains() to read the
live proxy config and discover which domains serve the management UI.
"""
from lib.nginx import get_management_domains as _resolve
return _resolve()
def is_domain_valid(domain: str) -> bool:
"""Check if *domain* is eligible for WebAuthn."""
return domain in get_management_domains()
def check_webauthn_config() -> None:
"""Validate WebAuthn config at startup."""
enabled = is_enabled()
if not enabled:
logger.info("WebAuthn is disabled in config")
return
try:
domains = get_management_domains()
if not domains:
logger.warning(
"WebAuthn is enabled but no management domains are configured. "
"Add a domain with backend 'webui' to nginx config, or disable WebAuthn."
)
else:
logger.info("WebAuthn enabled for domains: %s", domains)
except Exception as exc:
logger.error("Failed to resolve management domains for WebAuthn: %s", exc)
# ---------------------------------------------------------------------------
# Registration
# ---------------------------------------------------------------------------
def create_registration_options(
username: str,
origin: str,
rp_id: str,
) -> dict[str, Any]:
"""Create WebAuthn registration options for a new credential.
Args:
username: The user registering the credential.
origin: WebAuthn origin (must match the request origin).
rp_id: Relying Party ID (must match the request domain).
Returns a dict serializable to JSON, matching the format expected by
``navigator.credentials.create()``.
"""
# Load existing credential IDs to exclude
db = get_db()
rows = db.query(Q_SELECT_WEBAUTHN_USER, (username,))
exclude_credentials = [
PublicKeyCredentialDescriptor(id=b64u_decode(row["credential_id"]))
for row in rows
]
# Pad username to >= 8 bytes (required for user_id)
user_id = username.encode("utf-8")
if len(user_id) < 8:
user_id = user_id + b"\x00" * (8 - len(user_id))
options = generate_registration_options(
rp_id=rp_id,
rp_name=get_rp_name(),
user_name=username,
user_display_name=username,
user_id=user_id,
attestation=AttestationConveyancePreference.NONE,
authenticator_selection=AuthenticatorSelectionCriteria(
resident_key=ResidentKeyRequirement.PREFERRED,
user_verification=UserVerificationRequirement.PREFERRED,
),
supported_pub_key_algs=_SUPPORTED_ALGS,
exclude_credentials=exclude_credentials,
)
# Serialize using the library's built-in function
return json.loads(options_to_json(options))
def verify_registration(
username: str,
credential_response: dict[str, Any],
registration_options: dict[str, Any],
credential_name: str = "",
origin: str = "",
rp_id: str = "",
) -> dict[str, Any]:
"""Verify a registration response and persist the credential.
Args:
username: The user registering the credential.
credential_response: Browser response from ``credentials.create()``.
registration_options: The options dict from ``create_registration_options``.
credential_name: Optional human-readable label.
origin: WebAuthn origin for verification.
rp_id: Relying Party ID for verification.
Returns:
Dict with ``id``, ``name``, ``transports``, ``sign_count``.
"""
challenge = b64u_decode(registration_options["challenge"])
# The library accepts the credential response as a JSON-serializable dict
# We pass it directly — the library handles the parsing
col = verify_registration_response(
credential=credential_response,
expected_challenge=challenge,
expected_origin=origin,
expected_rp_id=rp_id,
require_user_verification=False,
)
new_cred = col.credential
new_credential_id = b64u_encode(new_cred.id)
new_public_key = b64u_encode(new_cred.public_key)
new_sign_count = new_cred.sign_count
transports = []
if hasattr(new_cred.response, "transports") and new_cred.response.transports:
transports = [str(t) for t in new_cred.response.transports]
if not transports:
transports_raw = credential_response.get("response", {}).get("transports", [])
transports = [
t
for t in transports_raw
if t
in (
"internal",
"hybrid",
"nfc",
"ble",
"usb",
"smart-card",
)
] or ["internal"]
db = get_db()
db.run(
Q_INSERT_WEBAUTHN,
(
username,
new_credential_id,
new_public_key,
new_sign_count,
credential_name,
json.dumps(transports),
),
)
logger.info(
"WebAuthn credential registered for %s: %s",
username,
credential_name or new_credential_id[:16],
)
return {
"id": new_credential_id,
"name": credential_name,
"transports": transports,
"sign_count": new_sign_count,
}
# ---------------------------------------------------------------------------
# Authentication
# ---------------------------------------------------------------------------
def create_authentication_options(
username: str,
rp_id: str,
) -> dict[str, Any] | None:
"""Create authentication options for a user.
Args:
username: The user authenticating.
rp_id: Relying Party ID (must match the request domain).
Returns a dict serializable to JSON (for ``navigator.credentials.get()``),
or ``None`` if the user has no registered credentials.
"""
db = get_db()
rows = db.query(Q_SELECT_WEBAUTHN_USER, (username,))
if not rows:
return None
allow_credentials = [
PublicKeyCredentialDescriptor(id=b64u_decode(row["credential_id"]))
for row in rows
]
options = generate_authentication_options(
rp_id=rp_id,
allow_credentials=allow_credentials,
user_verification=UserVerificationRequirement.PREFERRED,
)
return json.loads(options_to_json(options))
def verify_authentication(
username: str,
assertion_response: dict[str, Any],
auth_options: dict[str, Any],
origin: str = "",
rp_id: str = "",
) -> bool:
"""Verify an authentication assertion.
Args:
username: The user authenticating.
assertion_response: Browser response from ``credentials.get()``.
auth_options: The options dict from ``create_authentication_options``.
origin: WebAuthn origin for verification.
rp_id: Relying Party ID for verification.
Returns:
True on successful verification.
Raises:
ValueError: On verification failure.
"""
challenge = b64u_decode(auth_options["challenge"])
cred_id_str = assertion_response["id"]
# Load credential from DB
db = get_db()
cred_rows = db.query(Q_SELECT_WEBAUTHN_ID, (cred_id_str,))
if not cred_rows:
raise ValueError("Credential not found")
cred_row = cred_rows[0]
if cred_row["username"] != username:
raise ValueError("Credential not found")
public_key = b64u_decode(cred_row["public_key"])
old_sign_count = cred_row["sign_count"]
col = verify_authentication_response(
credential=assertion_response,
expected_challenge=challenge,
expected_origin=origin,
expected_rp_id=rp_id,
credential_public_key=public_key,
credential_current_sign_count=old_sign_count,
require_user_verification=False,
)
new_sign_count = col.credential_sign_count
if new_sign_count > old_sign_count:
db.run(
Q_UPDATE_WEBAUTHN_SIGN_COUNT,
(new_sign_count, cred_id_str),
)
logger.info("WebAuthn assertion verified for %s", username)
return True
# ---------------------------------------------------------------------------
# Credential management
# ---------------------------------------------------------------------------
def list_credentials(username: str) -> list[dict[str, Any]]:
"""List all registered credentials for a user."""
db = get_db()
rows = db.query(Q_SELECT_WEBAUTHN_USER, (username,))
result = []
for row in rows:
transports_str = row.get("transports", "[]")
try:
transports = json.loads(transports_str)
except (json.JSONDecodeError, TypeError):
transports = []
result.append(
{
"id": row["credential_id"],
"name": row.get("name") or "",
"transports": transports,
"sign_count": row.get("sign_count", 0),
}
)
return result
def remove_credential(username: str, credential_id: str) -> bool:
"""Remove a credential.
Raises:
ValueError: If credential not found or doesn't belong to user.
"""
db = get_db()
rows = db.query(Q_SELECT_WEBAUTHN_ID, (credential_id,))
if not rows:
raise ValueError(f"Credential {credential_id!r} not found")
if rows[0]["username"] != username:
raise ValueError(f"Credential {credential_id!r} not found")
db.run(Q_DELETE_WEBAUTHN, (credential_id,))
logger.info("WebAuthn credential removed: %s, %s", username, credential_id)
return True
def get_all_credential_counts() -> dict[str, int]:
"""Return credential counts for all users.
Returns:
Dict mapping usernames to credential counts.
"""
db = get_db()
rows = db.query(Q_SELECT_WEBAUTHN_COUNTS, ())
return {row["username"]: row["cred_count"] for row in rows}
+418 -86
View File
@@ -1,7 +1,8 @@
"""WireGuard Manager for Vacuum Wall SSL Proxy Firewall.
Generates wg-quick configurations, manages peers, and controls
the WireGuard tunnel interface.
the WireGuard tunnel interface. Supports multi-interface mode where
each access class gets its own WireGuard interface.
"""
import logging
@@ -19,7 +20,6 @@ logger = logging.getLogger(__name__)
PROJECT_DIR = Path(__file__).resolve().parent.parent
CONFIG_PATH = PROJECT_DIR / "config" / "wireguard" / "config.json"
WG_CONF_PATH = "/etc/wireguard/wg0.conf"
WG_QUICK_BIN = "wg-quick"
WG_BIN = "wg"
@@ -30,6 +30,25 @@ ENV = Environment(
trim_blocks=True,
)
def _wg_conf_path(ifname: str) -> str:
"""Return the system WG conf path for an interface name."""
return f"/etc/wireguard/{ifname}.conf"
def _default_class_fields() -> dict[str, Any]:
"""Return the default set of fields for an access class entry."""
return {
"name": "",
"description": "",
"subnet": None,
"listen_port": None,
"lan_access": False,
"private_key": "",
"public_key": "",
}
DEFAULT_CONFIG: dict[str, Any] = {
"interface": {
"name": "wg0",
@@ -37,9 +56,31 @@ DEFAULT_CONFIG: dict[str, Any] = {
"private_key": "",
"public_key": "",
"addresses": ["10.137.0.1/24"],
"server_endpoint": "",
"description": "",
"post_up": None,
"post_down": None,
},
"access_classes": {
"full": {
"name": "Full LAN Access",
"description": "Peers get full access to internal networks",
"subnet": "10.137.0.0/24",
"listen_port": 51820,
"lan_access": True,
"private_key": "",
"public_key": "",
},
"internet": {
"name": "Internet Only",
"description": "Peers can only reach the internet",
"subnet": "10.137.1.0/24",
"listen_port": 51821,
"lan_access": False,
"private_key": "",
"public_key": "",
},
},
"peers": {},
}
@@ -62,23 +103,114 @@ def save_config(cfg: dict[str, Any]) -> None:
def generate_keypair() -> tuple[str, str]:
"""Generate a WireGuard private/public key pair using ``wg`` CLI."""
res = run_proc([WG_BIN, "genkey"], sudo=True)
res = run_proc([WG_BIN, "genkey"], sudo=False)
private_key = res.stdout.strip()
res2 = run_proc([WG_BIN, "pubkey"], sudo=True, input=private_key)
res2 = run_proc([WG_BIN, "pubkey"], sudo=False, input=private_key)
public_key = res2.stdout.strip()
return private_key, public_key
# --- wg0.conf generation ---
# --- Helpers ---
def _class_interface_name(class_key: str) -> str:
"""Derive interface name for an access class key."""
return f"wg-{class_key}"
def _class_zone_name(class_key: str) -> str:
"""Derive firewall zone name for an access class key."""
return f"vpn-{class_key}"
def get_class_interface_name(class_key: str) -> str:
"""Public wrapper for `_class_interface_name`."""
return _class_interface_name(class_key)
def get_class_zone_name(class_key: str) -> str:
"""Public wrapper for `_class_zone_name`."""
return _class_zone_name(class_key)
def _class_peers(cfg: dict[str, Any], class_key: str) -> dict[str, Any]:
"""Return peers assigned to a given access class."""
return {
name: info
for name, info in cfg.get("peers", {}).items()
if isinstance(info, dict) and info.get("access_class") == class_key
}
# --- wg-<class>.conf generation ---
def generate_class_conf(cfg: dict[str, Any], class_key: str) -> str | None:
"""Render a wg-quick config file for a single access class.
Returns ``None`` when the class has no peers assigned.
"""
classes = cfg.get("access_classes", {})
class_cfg = classes.get(class_key)
if not class_cfg or not isinstance(class_cfg, dict):
return None
peers = _class_peers(cfg, class_key)
if not peers:
return None
ifname = _class_interface_name(class_key)
subnet = class_cfg.get("subnet")
if not subnet:
subnet = "10.137.0.0/24"
_, prefix = subnet.rsplit("/", 1)
base = subnet.rsplit(".", 1)[0]
addr = f"{base}.1/{prefix}"
listen_port = class_cfg.get("listen_port")
if not listen_port:
listen_port = 51820
private_key = class_cfg.get("private_key", "")
if not private_key:
raise ValueError(
f"Access class '{class_key}' has no private key — generate one first."
)
class_iface = {
"name": ifname,
"listen_port": listen_port,
"private_key": private_key,
"addresses": [addr],
"post_up": cfg.get("interface", {}).get("post_up"),
"post_down": cfg.get("interface", {}).get("post_down"),
}
tmpl = ENV.get_template("wireguard.conf")
return tmpl.render(
timestamp=datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
interface=class_iface,
peers=peers,
)
def generate_conf(cfg: dict[str, Any]) -> str:
"""Render a valid wg-quick config file from *cfg* using Jinja2."""
"""Render a valid wg-quick config file from *cfg* using Jinja2.
Legacy single-interface mode uses the top-level ``interface`` block
and peers without an ``access_class`` assigned.
"""
fallback_peers = {
n: p
for n, p in cfg.get("peers", {}).items()
if isinstance(p, dict) and not p.get("access_class")
}
tmpl = ENV.get_template("wireguard.conf")
return tmpl.render(
timestamp=datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ"),
interface=cfg["interface"],
peers=cfg.get("peers", {}),
peers=fallback_peers if fallback_peers else {},
)
@@ -86,55 +218,191 @@ def generate_conf(cfg: dict[str, Any]) -> str:
def apply() -> None:
"""Write the current config to disk and bring the tunnel up with wg-quick."""
"""Write the current config to disk and bring tunnels up with wg-quick.
In multi-interface mode, applies each access class's interface independently.
Falls back to legacy single-interface mode when no classes have peers.
"""
cfg = get_config()
classes = cfg.get("access_classes", {})
applied = False
for class_key in classes:
class_cfg = classes.get(class_key)
if not class_cfg or not isinstance(class_cfg, dict):
continue
if not _class_peers(cfg, class_key):
continue
try:
conf_text = generate_class_conf(cfg, class_key)
if not conf_text:
continue
except ValueError:
continue
ifname = _class_interface_name(class_key)
conf_path = _wg_conf_path(ifname)
local_dir = PROJECT_DIR / "data" / "wireguard"
local_dir.mkdir(parents=True, exist_ok=True)
local_tmp = local_dir / f"{ifname}.conf.tmp"
with open(local_tmp, "w") as f:
f.write(conf_text)
os.chmod(local_tmp, 0o600)
run(["cp", "--", str(local_tmp), conf_path], sudo=True)
run(["chown", "root:root", conf_path], sudo=True, check=False)
local_tmp.unlink(missing_ok=True)
run([WG_QUICK_BIN, "up", ifname], sudo=True)
logger.info("WireGuard tunnel '%s' (class '%s') brought up", ifname, class_key)
applied = True
if applied:
save_config(cfg)
elif applied is False:
# Legacy single-interface fallback
legacy_apply(cfg)
def legacy_apply(cfg: dict[str, Any]) -> None:
"""Legacy single-interface apply."""
conf_text = generate_conf(cfg)
save_config(cfg)
ifname = cfg["interface"]["name"]
conf_path = _wg_conf_path(ifname)
local_dir = PROJECT_DIR / "data" / "wireguard"
local_dir.mkdir(parents=True, exist_ok=True)
local_tmp = local_dir / "wg0.conf.tmp"
local_tmp = local_dir / f"{ifname}.conf.tmp"
with open(local_tmp, "w") as f:
f.write(conf_text)
os.chmod(local_tmp, 0o600)
run(["cp", "--", str(local_tmp), WG_CONF_PATH], sudo=True)
run(["chown", "root:root", WG_CONF_PATH], sudo=True, check=False)
run(["cp", "--", str(local_tmp), conf_path], sudo=True)
run(["chown", "root:root", conf_path], sudo=True, check=False)
local_tmp.unlink(missing_ok=True)
run([WG_QUICK_BIN, "up", ifname], sudo=True)
logger.info("WireGuard tunnel '%s' brought up", ifname)
run([WG_QUICK_BIN, "up", cfg["interface"]["name"]], sudo=True)
logger.info("WireGuard tunnel '%s' brought up", cfg["interface"]["name"])
def apply_class(class_key: str) -> None:
"""Apply config for a single access class interface."""
cfg = get_config()
class_cfg = cfg.get("access_classes", {}).get(class_key)
if not class_cfg or not isinstance(class_cfg, dict):
raise ValueError(f"Access class '{class_key}' not found")
conf_text = generate_class_conf(cfg, class_key)
if not conf_text:
raise ValueError(f"No peers assigned to class '{class_key}'")
ifname = _class_interface_name(class_key)
conf_path = _wg_conf_path(ifname)
local_dir = PROJECT_DIR / "data" / "wireguard"
local_dir.mkdir(parents=True, exist_ok=True)
local_tmp = local_dir / f"{ifname}.conf.tmp"
with open(local_tmp, "w") as f:
f.write(conf_text)
os.chmod(local_tmp, 0o600)
run(["cp", "--", str(local_tmp), conf_path], sudo=True)
run(["chown", "root:root", conf_path], sudo=True, check=False)
local_tmp.unlink(missing_ok=True)
run([WG_QUICK_BIN, "up", ifname], sudo=True)
save_config(cfg)
logger.info("WireGuard tunnel '%s' (class '%s') brought up", ifname, class_key)
def down() -> None:
"""Bring the WireGuard tunnel interface down."""
"""Bring all WireGuard tunnel interfaces down.
In multi-interface mode, brings down each class interface with peers.
"""
cfg = get_config()
name = cfg["interface"]["name"]
run([WG_QUICK_BIN, "down", name], sudo=True)
logger.info("WireGuard tunnel '%s' brought down", name)
classes = cfg.get("access_classes", {})
for class_key in classes:
class_cfg = classes.get(class_key)
if not class_cfg or not isinstance(class_cfg, dict):
continue
if ifname := _class_interface_name(class_key):
try:
run([WG_QUICK_BIN, "down", ifname], sudo=True)
logger.info("WireGuard tunnel '%s' brought down", ifname)
except Exception:
pass
# Also try legacy interface (skip if name matches any class interface)
ifname = cfg["interface"].get("name", "")
if ifname:
class_names = {_class_interface_name(k) for k in classes}
if ifname not in class_names:
try:
run([WG_QUICK_BIN, "down", ifname], sudo=True)
logger.info("WireGuard tunnel '%s' brought down", ifname)
except Exception:
pass
def down_class(class_key: str) -> None:
"""Bring down a single access class interface."""
ifname = _class_interface_name(class_key)
run([WG_QUICK_BIN, "down", ifname], sudo=True)
logger.info("WireGuard tunnel '%s' brought down", ifname)
# --- Status ---
def status() -> dict[str, Any]:
"""Query the live tunnel state via ``wg show``."""
"""Query the live tunnel state via ``wg show``.
In multi-interface mode, collects status for all class interfaces.
Returns combined status dict keyed by interface name.
"""
cfg = get_config()
name = cfg["interface"]["name"]
result: dict[str, Any] = {
"up": False,
"interface": {},
"peers": [],
"classes": {},
}
# Collect per-class status
classes = cfg.get("access_classes", {})
for class_key in classes:
ifname = _class_interface_name(class_key)
try:
res = run_proc([WG_BIN, "show", ifname], sudo=True, check=False)
if res.returncode != 0:
result["classes"][class_key] = {"up": False, "peers": []}
continue
class_status = parse_wg_show_output(res.stdout.strip())
result["classes"][class_key] = class_status
if class_status["up"]:
result["up"] = True
except Exception:
result["classes"][class_key] = {"up": False, "peers": []}
# Legacy single-interface status (still collected for backward compat)
try:
ifname = cfg["interface"].get("name", "wg0")
res = run_proc([WG_BIN, "show", ifname], sudo=True, check=False)
if res.returncode == 0:
parsed = parse_wg_show_output(res.stdout.strip())
result["up"] = parsed["up"]
result["interface"] = parsed.get("interface", {})
result["peers"] = parsed.get("peers", [])
except Exception:
pass
return result
def parse_wg_show_output(raw: str) -> dict[str, Any]:
"""Parse ``wg show`` output into structured dict.
Returns ``{"up", "interface", "peers"}`` where *interface* carries
``public_key``, ``listen_port`` and (when present) ``fwmark``.
"""
result: dict[str, Any] = {
"up": False,
"interface": {},
"peers": [],
}
try:
res = run_proc([WG_BIN, "show", name], sudo=True, check=False)
if res.returncode != 0:
return result
raw = res.stdout.strip()
except Exception:
return result
current_peer: dict[str, Any] | None = None
peers: list[dict[str, Any]] = []
@@ -169,8 +437,8 @@ def status() -> dict[str, Any]:
"endpoint": None,
"allowed_ips": [],
"latest_handshake": None,
"transfer_received": 0,
"transfer_sent": 0,
"transfer_received": "0",
"transfer_sent": "0",
"persistent_keepalive": None,
}
peers.append(current_peer)
@@ -214,36 +482,50 @@ def status() -> dict[str, Any]:
# --- Peer management ---
_UNSET = object()
def add_peer(
name: str,
endpoint: str | None = None,
allowed_ips: list[str] | None = None,
persistent_keepalive: int | None = None,
endpoint: str | None | object = _UNSET,
allowed_ips: list[str] | None | object = _UNSET,
persistent_keepalive: int | None | object = _UNSET,
preshared_key: str | None = None,
description: str | None = None,
access_class: str | None = None,
) -> dict[str, Any]:
"""Add (or update) a peer in the configuration."""
cfg = get_config()
peers = cfg.setdefault("peers", {})
allowed_ips = allowed_ips or []
if name in peers:
peer = peers[name]
peer["endpoint"] = endpoint
peer["allowed_ips"] = allowed_ips
peer["persistent_keepalive"] = persistent_keepalive
if endpoint is not _UNSET:
peer["endpoint"] = endpoint
if allowed_ips is not _UNSET:
peer["allowed_ips"] = allowed_ips if allowed_ips is not None else []
if persistent_keepalive is not _UNSET:
peer["persistent_keepalive"] = persistent_keepalive
if preshared_key is not None:
peer["preshared_key"] = preshared_key
if description is not None:
peer["description"] = description
if access_class is not None:
peer["access_class"] = access_class
logger.info("WireGuard peer '%s' updated", name)
else:
priv, pub = generate_keypair()
peer = {
"public_key": pub,
"private_key": priv,
"endpoint": endpoint,
"allowed_ips": allowed_ips,
"persistent_keepalive": persistent_keepalive,
"endpoint": endpoint if endpoint is not _UNSET else None,
"allowed_ips": allowed_ips if allowed_ips is not _UNSET else [],
"persistent_keepalive": persistent_keepalive
if persistent_keepalive is not _UNSET
else None,
"preshared_key": preshared_key,
"description": description,
"access_class": access_class,
}
peers[name] = peer
logger.info("WireGuard peer '%s' added (pubkey=%s...)", name, pub[:16])
@@ -275,9 +557,17 @@ def get_peers() -> list[dict[str, Any]]:
def get_peer_status() -> list[dict[str, Any]]:
"""Return live peer status from ``wg show``."""
"""Return live peer status from ``wg show`` for all interfaces."""
st = status()
return st.get("peers", [])
all_peers: list[dict[str, Any]] = []
for _class_key, class_st in st.get("classes", {}).items():
for p in class_st.get("peers", []):
merged = dict(p)
merged["access_class"] = _class_key
all_peers.append(merged)
if not all_peers:
all_peers = st.get("peers", [])
return all_peers
# --- Client config generation ---
@@ -286,9 +576,13 @@ def get_peer_status() -> list[dict[str, Any]]:
def generate_client_conf(
peer_name: str,
server_endpoint: str,
server_pubkey: str,
server_pubkey: str | None = None,
) -> str:
"""Build a client-side wg-quick config snippet for *peer_name*."""
"""Build a client-side wg-quick config snippet for *peer_name*.
Uses the peer's access class to derive server address from the class
subnet and the class's listen port.
"""
cfg = get_config()
iface = cfg["interface"]
peer = cfg["peers"].get(peer_name)
@@ -301,12 +595,30 @@ def generate_client_conf(
f"Peer '{peer_name}' has no private key — cannot generate client config."
)
# Determine class info for address/port
access_class = peer.get("access_class")
sorted_peers = sorted(cfg.get("peers", {}).keys())
peer_index = sorted_peers.index(peer_name) + 2
srv_addr = iface["addresses"][0] if iface["addresses"] else "10.137.0.1/24"
addr_part, prefix = srv_addr.rsplit("/", 1)
prefix_base = addr_part.rsplit(".", 1)[0]
client_addr = f"{prefix_base}.{peer_index}/{prefix}"
if access_class and access_class in cfg.get("access_classes", {}):
class_cfg = cfg["access_classes"][access_class]
subnet = class_cfg.get("subnet", "10.137.0.0/24")
listen_port = class_cfg.get("listen_port", 51820)
else:
subnet = iface.get("addresses", ["10.137.0.1/24"])[0]
listen_port = iface.get("listen_port", 51820)
if "/" not in subnet:
subnet = f"{subnet}/24"
addr = subnet.rsplit(".", 1)[0]
prefix = subnet.rsplit("/", 1)[1]
client_addr = f"{addr}.{peer_index}/{prefix}"
sk = server_pubkey or iface.get("public_key", "")
ep = server_endpoint or iface.get("server_endpoint", "")
if ep and listen_port:
host = ep.split(":")[0]
ep = f"{host}:{listen_port}"
tmpl = ENV.get_template("wireguard-client.conf")
conf = tmpl.render(
@@ -314,8 +626,8 @@ def generate_client_conf(
peer_name=peer_name,
client_priv=client_priv,
client_addr=client_addr,
server_pubkey=server_pubkey,
server_endpoint=server_endpoint,
server_pubkey=sk,
server_endpoint=ep,
allowed_ips=peer.get("allowed_ips", ["0.0.0.0/0"]),
preshared_key=peer.get("preshared_key"),
persistent_keepalive=peer.get("persistent_keepalive"),
@@ -351,70 +663,90 @@ def set_post_down(cmd: str | None) -> None:
save_config(cfg)
# --- Class key generation ---
def generate_class_keypair(class_key: str) -> tuple[str, str]:
"""Generate a key pair for an access class interface."""
cfg = get_config()
classes = cfg.setdefault("access_classes", {})
if class_key not in classes:
raise ValueError(f"Access class '{class_key}' not found")
class_cfg = classes[class_key]
if class_cfg.get("private_key"):
return class_cfg["private_key"], class_cfg["public_key"]
priv, pub = generate_keypair()
class_cfg["private_key"] = priv
class_cfg["public_key"] = pub
save_config(cfg)
logger.info("Key pair generated for class '%s'", class_key)
return priv, pub
# --- Initialise ---
def _ensure_class_defaults(cfg: dict[str, Any]) -> None:
"""Ensure access classes have required Phase-2 fields."""
classes = cfg.setdefault("access_classes", {})
for _key, c in classes.items():
if not isinstance(c, dict):
continue
for field, default in _default_class_fields().items():
if field not in c:
c[field] = default
def _ensure_access_classes(cfg: dict[str, Any]) -> None:
"""Pre-seed default access classes if missing or empty (idempotent).
Also upgrades existing classes with Phase-2 fields.
"""
classes = cfg.setdefault("access_classes", {})
if not classes:
full_defaults = deepcopy(DEFAULT_CONFIG["access_classes"])
classes.update(full_defaults)
return
_ensure_class_defaults(cfg)
def initialize() -> dict[str, Any]:
"""Perform first-time WireGuard setup."""
cfg = get_config()
if cfg["interface"].get("private_key"):
_ensure_access_classes(cfg)
save_config(cfg)
return cfg
priv, pub = generate_keypair()
cfg["interface"]["private_key"] = priv
cfg["interface"]["public_key"] = pub
_ensure_access_classes(cfg)
save_config(cfg)
logger.info("WireGuard initialised (pubkey=%s...)", pub[:16])
return cfg
# --- Utility: parse wg show into structured peer map ---
def _parse_wg_show(output: str) -> dict[str, Any]:
"""Internal parser for ``wg show`` multiline output."""
peers: dict[str, dict[str, Any]] = {}
current: dict[str, Any] | None = None
for line in output.splitlines():
line = line.strip()
if line.startswith("peer:"):
key = line.split(":", 1)[1].strip()
current = {"_key": key}
peers[key] = current
continue
if current is None:
continue
if line.startswith("endpoint:"):
val = line.split(":", 1)[1].strip()
current["endpoint"] = val
elif line.startswith("allowed ips:"):
current["allowed_ips"] = line.split(":", 1)[1].strip()
elif line.startswith("latest handshake:"):
current["latest_handshake"] = line.split(":", 1)[1].strip()
elif line.startswith("transfer:"):
current["transfer_raw"] = line.split(":", 1)[1].strip()
elif line.startswith("persistent-keepalive:"):
current["persistent_keepalive"] = line.split(":", 1)[1].strip()
return peers
__all__ = [
"DEFAULT_CONFIG",
"add_peer",
"apply",
"apply_class",
"down",
"down_class",
"generate_class_conf",
"generate_class_keypair",
"generate_client_conf",
"generate_conf",
"generate_keypair",
"get_class_interface_name",
"get_class_zone_name",
"get_config",
"get_peer_status",
"get_peers",
"initialize",
"parse_wg_show_output",
"remove_peer",
"save_config",
"set_listen_port",
+3
View File
@@ -12,6 +12,9 @@ dependencies = [
"aiohttp>=3.9,<4.0",
"passlib>=1.7.4,<2.0",
"requests-unixsocket>=0.2,<1.0",
"PyJWT>=2.8,<3.0",
"argon2-cffi>=23.1.0,<25.0",
"webauthn>=3.0.0,<4.0",
]
[project.scripts]
+95
View File
@@ -0,0 +1,95 @@
"""Bootstrap auth: initialize DB and seed admin user at install time.
Idempotent safe to run on every install (and re-install):
- Writes config/auth/config.json only if it does not exist (existing
JWT/WebAuthn settings are preserved).
- Creates the admin user if missing; if the user already exists, updates
the admin password to the provided value (docs/deployment.md: "On
re-run, updates the admin password if already present").
- Suppresses the last-resort builtin admin seed (VACUUM_WALL_SEED_BUILTIN_ADMIN=0):
bootstrap is the operator user's creator on a fresh install, so exactly
one account exists and no hardcoded admin with an unrecoverable random
password is left behind.
Usage:
python scripts/bootstrap_auth.py --project-dir /path/to/project \
--username admin \
--password secret \
--domain wall.example.com
"""
import argparse
import json
import os
import sys
from pathlib import Path
# Ensure project lib is importable
def main() -> None:
parser = argparse.ArgumentParser(description="Bootstrap Vacuum Wall auth")
parser.add_argument("--project-dir", required=True, help="Project root directory")
parser.add_argument("--username", required=True, help="Admin username")
parser.add_argument("--password", required=True, help="Admin password")
parser.add_argument("--domain", required=True, help="Management domain (rp_id)")
args = parser.parse_args()
project_dir = Path(args.project_dir).resolve()
sys.path.insert(0, str(project_dir))
# Set DB path before importing lib modules
db_path = str(project_dir / "data" / "auth.db")
os.environ["VACUUM_WALL_DB_BACKEND"] = "sqlite"
os.environ["VACUUM_WALL_DB_PATH"] = db_path
# Suppress the last-resort builtin admin seed in get_db(): bootstrap
# creates the operator user itself, so no seeded admin may shadow it.
os.environ["VACUUM_WALL_SEED_BUILTIN_ADMIN"] = "0"
from lib.auth_users import (
ALL_SUBSYSTEMS,
create_user,
find_user,
reset_password,
)
# Write config — only if missing, so re-runs never clobber existing
# JWT/WebAuthn settings (e.g. a customized rp_id/origin).
config_dir = project_dir / "config" / "auth"
config_dir.mkdir(parents=True, exist_ok=True)
config_path = config_dir / "config.json"
if not config_path.exists():
config = {
"jwt": {
"access_token_ttl": 300,
"refresh_token_ttl": 604800,
"algorithm": "HS256",
},
"webauthn": {
"rp_name": "Vacuum Wall",
"rp_id": args.domain,
"origin": f"https://{args.domain}",
},
}
with open(config_path, "w") as f:
json.dump(config, f, indent=2)
f.write("\n")
print(f"Wrote auth config: {config_path}")
else:
print(f"Auth config already present, leaving unchanged: {config_path}")
# Initialize DB and create the admin user, or sync the password on re-run
if find_user(args.username) is not None:
reset_password(args.username, args.password)
print(f"Updated existing user: {args.username} (password synced)")
else:
permissions = {sub: "rw" for sub in ALL_SUBSYSTEMS}
user = create_user(args.username, args.password, permissions)
print(f"Created admin user: {user['username']} (id={user['id']})")
print(f"Permissions: {len(permissions)} subsystems, all 'rw'")
if __name__ == "__main__":
main()
+156 -134
View File
@@ -35,14 +35,14 @@ while [[ $# -gt 0 ]]; do
--lan-ifaces) _cli_lan_ifaces="$2"; shift 2 ;;
-h|--help)
printf '%s\n' \
"Usage: install.sh [OPTIONS]" \
"Usage: scripts/install.sh [OPTIONS]" \
"" \
"Options:" \
" --user, -u USER WebUI user (created if it does not exist, required for non-dev mode)" \
" --path, -p DIR Install directory (default: repo root)" \
" --dev Dev mode: auto-detect repo owner, skip safety warning" \
" --mgmt-pass PASS WebUI basic auth password (required)" \
" --mgmt-user USER WebUI basic auth username (default: admin)" \
" --mgmt-pass PASS Initial admin password (required)" \
" --mgmt-user USER Initial admin username (default: admin)" \
" --mgmt-domain DOMAIN Management domain (auto-detected)" \
" --wan-iface IFACE WAN interface name (auto-detected)" \
" --lan-ifaces IFC,... LAN interface names, comma-separated (auto-detected)" \
@@ -54,10 +54,10 @@ while [[ $# -gt 0 ]]; do
" CLI flags take precedence over env vars." \
"" \
"Example (dev):" \
" ./install.sh --dev --mgmt-pass pass" \
" ./scripts/install.sh --dev --mgmt-pass pass" \
"" \
"Example (prod):" \
" MGMT_PASS=pass ./install.sh --user vacuum-wall"
" MGMT_PASS=pass ./scripts/install.sh --user vacuum-wall"
exit 0
;;
*)
@@ -67,11 +67,11 @@ while [[ $# -gt 0 ]]; do
done
# --- Resolve config: CLI flag > env var > default ---
REPO_DIR="$(cd "$(dirname "$0")" && pwd)"
REPO_DIR="$(cd "$(dirname "$0")/../" && pwd)"
# Required settings (no defaults — must be provided)
MGMT_PASS="${_cli_mgmt_pass:-${MGMT_PASS:-}}"
# Optional settings with defaults
# MGMT_PASS is strictly required — admin user is created at install time
MGMT_PASS="${_cli_mgmt_pass:-${MGMT_PASS:-}}"
MGMT_USER="${_cli_mgmt_user:-${MGMT_USER:-admin}}"
# MGMT_DOMAIN — CLI > env > auto-detect from hostname
@@ -104,19 +104,7 @@ LAN_IFACES="${_cli_lan_ifaces:-${LAN_IFACES:-}}"
[[ -f /etc/debian_version ]] || warn "This script is designed for Debian/Ubuntu."
# --- Validate required settings ---
missing=()
[[ -z "$MGMT_PASS" ]] && missing+=("MGMT_PASS (--mgmt-pass)")
if (( ${#missing[@]} )); then
echo -e "${RED}[!!]${NC} Missing required settings:"
for v in "${missing[@]}"; do
case "$v" in
"MGMT_PASS (--mgmt-pass)") echo ' export MGMT_PASS="your-password" # or --mgmt-pass';;
esac
done
printf '\nTo run: MGMT_PASS=pass ./install.sh\n'
exit 1
fi
[[ -n "$MGMT_PASS" ]] || err "MGMT_PASS is required (set --mgmt-pass or MGMT_PASS env var)"
ACME_HOME="$PROJECT_DIR/data/acme"
# Dev mode: auto-detect repo owner as service user
@@ -156,6 +144,32 @@ fi
# Shared group: use the WebUI user's primary group
USER_GROUP=$(id -gn "$USER_NAME")
# Some appliance images ship with top-level system directories (and sometimes
# everything under them) owned by a regular user. This trips systemd-tmpfiles'
# "unsafe path transition" check and lets that user modify system paths.
# Repair the top level here; warn with a full-repair command if deeper
# mis-ownership is detected (depth-1 entries of /etc /usr /var /boot are
# always root-owned on Debian, so this check cannot false-positive).
_sys_dirs=(/ /bin /boot /etc /home /media /mnt /opt /root /sbin /srv /usr /var /var/lib /var/log)
_misowned=()
for _d in "${_sys_dirs[@]}"; do
[[ -e "$_d" ]] || continue
[[ "$(stat -c '%U' "$_d" 2>/dev/null)" == "root" ]] || _misowned+=("$_d")
done
if [[ ${#_misowned[@]} -gt 0 ]]; then
warn "System directories not owned by root: ${_misowned[*]}"
warn "Chowning to root:root (image shipped with mis-owned system paths)."
chown root:root "${_misowned[@]}"
_deep_count=$(find /etc /usr /var /boot -maxdepth 1 ! -user root 2>/dev/null | wc -l)
if [[ "$_deep_count" -gt 0 ]]; then
warn "Deeper mis-ownership detected ($_deep_count entries at depth 1)."
warn "Run a full repair, then re-run this installer:"
warn " sudo find / -xdev -path /proc -prune -o -path /sys -prune -o -path /dev -prune -o -path /run -prune -o -path /tmp -prune -o -path /home/$USER_NAME -prune -o -user $USER_NAME -print0 | xargs -0 -r chown root:root"
else
log "Repaired top-level system directory ownership."
fi
fi
echo "============================================"
echo " Vacuum Wall Appliance Installer"
echo " Install dir: $PROJECT_DIR"
@@ -182,6 +196,10 @@ apt-get install -y -qq \
apache2-utils \
avahi-daemon
# --- 1b. Vendored libraries ---
log "Downloading vendored libraries..."
bash "${PROJECT_DIR}/scripts/update-vendor.sh" || err "update-vendor.sh failed"
# --- 2. Setup users ---
log "WebUI user: $USER_NAME (group: $USER_GROUP)"
@@ -207,23 +225,22 @@ else
chmod -R g+x "${PROJECT_DIR}/.venv"
fi
# --- 2c. Install acme.sh (vendored) ---
if [[ ! -x "$ACME_HOME/acme.sh" ]]; then
log "Installing acme.sh (vendored)..."
mkdir -p "$ACME_HOME"
cp "${PROJECT_DIR}/vendor/acme.sh" "$ACME_HOME/acme.sh"
chmod +x "$ACME_HOME/acme.sh"
chown -R "$USER_DAEMON_NAME:$USER_GROUP" "$ACME_HOME"
else
log "acme.sh already installed."
fi
# Install the deploy hook into acme.sh's deploy directory
# (acme.sh only resolves hooks from $ACME_HOME/deploy/)
mkdir -p "$ACME_HOME/deploy"
cp "${PROJECT_DIR}/system/acme-deploy.sh" "$ACME_HOME/deploy/acme-deploy.sh"
chown "$USER_DAEMON_NAME:$USER_GROUP" "$ACME_HOME/deploy/acme-deploy.sh"
chmod 0755 "$ACME_HOME/deploy/acme-deploy.sh"
# Ensure the daemon user owns acme.sh's runtime conf files (account.conf and
# any per-domain .conf). acme.sh hardens these owner-only (600); if a
# non-daemon user ever (re)creates them the daemon cannot source account.conf
# and every acme.sh call exits 2. The daemon self-heals on the next run, but
# fixing ownership here avoids the initial broken window on fresh installs.
if [ -d "$ACME_HOME" ]; then
find "$ACME_HOME" -maxdepth 1 -type f -name '*.conf' \
-exec chown "$USER_DAEMON_NAME:$USER_GROUP" {} + 2>/dev/null || true
[ -f "$ACME_HOME/account.conf" ] && chmod 0640 "$ACME_HOME/account.conf"
fi
# --- 3. Setup directories ---
log "Creating config and data directories..."
@@ -231,15 +248,35 @@ mkdir -p "${PROJECT_DIR}/config"/{dnsmasq,nginx,wireguard,firewall}
mkdir -p "${PROJECT_DIR}/data"/{nginx/sites-enabled,dnsmasq,firewall,wireguard,acme}
mkdir -p /etc/wireguard
mkdir -p /etc/dnsmasq
# Set ownership: daemon owns project dir in prod, repo owner keeps ownership in dev
# nginx workers (www-data) serve webui/static directly from disk for the
# management domain — ensure read access regardless of checkout umask.
chmod -R a+rX "${PROJECT_DIR}/webui/static"
# ...and traversal (x only) up the parent chain, so repo-in-$HOME installs work.
_d="${PROJECT_DIR}"
while [[ "$_d" != "/" && -n "$_d" ]]; do
chmod a+x "$_d" 2>/dev/null || true
_d="$(dirname "$_d")"
done
# Set ownership: daemon owns project dir in prod, repo owner keeps ownership in dev.
# The top-level .git (directory or worktree pointer file) is left untouched so
# the repo owner's git isn't tripped by git's dubious-ownership check.
if [[ "$_cli_is_dev" == true ]]; then
_dev_owner="$USER_NAME"
else
_dev_owner="$USER_DAEMON_NAME"
fi
chown -R "$_dev_owner:$USER_GROUP" "$PROJECT_DIR"
chmod -R g+rwX "$PROJECT_DIR"
find "$PROJECT_DIR" -type d -exec chmod g+s '{}' +
(
shopt -s dotglob nullglob
for _entry in "$PROJECT_DIR"/*; do
[[ "$(basename "$_entry")" == ".git" ]] && continue
chown -R "$_dev_owner:$USER_GROUP" "$_entry"
chmod -R g+rwX "$_entry"
find "$_entry" -type d -exec chmod g+s '{}' +
done
# Top dir: ownership + shared-group access (never .git)
chown "$_dev_owner:$USER_GROUP" "$PROJECT_DIR"
chmod g+rwX,g+s "$PROJECT_DIR"
)
# --- 4. Template rendering function ---
# Renders Jinja2 templates by injecting env vars as template context.
@@ -282,13 +319,23 @@ render_template "${PROJECT_DIR}/system/systemd/vacuum-wall-acme.service" \
| install -m 0644 /dev/stdin /etc/systemd/system/vacuum-wall-acme.service
install -m 0644 "${PROJECT_DIR}/system/systemd/vacuum-wall-acme.timer" /etc/systemd/system/vacuum-wall-acme.timer
# Volatile /run entries (sudo, firewalld) must exist before vacuum-walld
# spawns — systemd-tmpfiles-setup.service restores them at every boot.
install -m 0644 "${PROJECT_DIR}/system/tmpfiles.d/vacuum-wall.conf" /etc/tmpfiles.d/vacuum-wall.conf
systemd-tmpfiles --create
systemctl daemon-reload
# --- 7. Enable IP forwarding (persistent via sysctl.conf) ---
# --- 7. Enable IP forwarding (persistent via sysctl.conf + runtime apply) ---
log "Enabling IP forwarding..."
if ! grep -q "^net.ipv4.ip_forward=1" /etc/sysctl.conf 2>/dev/null; then
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
fi
# Apply immediately so NAT works without reboot
if [ "$(cat /proc/sys/net/ipv4/ip_forward 2>/dev/null)" != "1" ]; then
sysctl -w net.ipv4.ip_forward=1 >/dev/null 2>&1 && log "IP forwarding enabled at runtime" || \
warn "Could not enable IP forwarding at runtime"
fi
# --- 8. Detect network interfaces ---
log "Detecting network interfaces..."
@@ -355,108 +402,87 @@ else
chown "$USER_DAEMON_NAME:$USER_GROUP" "$_SOCKET" 2>/dev/null || true
chmod 0660 "$_SOCKET" 2>/dev/null || true
# --- 10. Configure subsystems via daemon API ---
log "Configuring subsystems via daemon API..."
WAN_IFACE="$WAN_IFACE" \
LAN_IFACES="$LAN_IFACES" \
MGMT_DOMAIN="$DOMAIN" \
MGMT_USER="$MGMT_USER" \
MGMT_PASS="$MGMT_PASS" \
"${PROJECT_DIR}/.venv/bin/python3" -c "
import daemon.client as c
from daemon.iface import (
POST_ACME_SELF_SIGNED, POST_NGINX_DOMAINS_ADD, POST_NGINX_APPLY,
POST_FIREWALL_CONFIG, POST_FIREWALL_CONFIG_APPLY, POST_NETWORK_SYSCTL_SET,
GET_NETWORK_INFER_DHCP_RANGES,
)
import sys
echo ""
echo " Setting up initial management configuration..."
domain = '${DOMAIN}'
mgmt_user = '${MGMT_USER}'
mgmt_pass = '${MGMT_PASS}'
wan_iface = '${WAN_IFACE}'
lan_ifaces = '${LAN_IFACES}'
# Bootstrap auth: generate config + seed admin user. bootstrap_auth.py
# is idempotent — on re-run it preserves the existing config and
# updates the admin password to MGMT_PASS (docs/deployment.md).
if [[ ! -f "${PROJECT_DIR}/config/auth/config.json" ]]; then
echo ""
echo " Bootstrapping auth (creating admin user: $MGMT_USER)..."
else
echo ""
echo " Syncing admin password for existing user: $MGMT_USER..."
fi
# Self-signed cert for management domain
try:
res = c.post(POST_ACME_SELF_SIGNED, {'domain': domain, 'days': 365})
print(f' [cert] Self-signed: {\"generated\" if res.get(\"generated\") else \"exists\"}')
except Exception as e:
print(f' [cert] Warning: {e}', file=sys.stderr)
"${PROJECT_DIR}/.venv/bin/python3" "${PROJECT_DIR}/scripts/bootstrap_auth.py" \
--project-dir "$PROJECT_DIR" \
--username "$MGMT_USER" \
--password "$MGMT_PASS" \
--domain "$DOMAIN"
# Management proxy domain + htpasswd
try:
c.post(POST_NGINX_DOMAINS_ADD, {
'domain': domain,
'paths': {
'/': {
'backend': {'host': '127.0.0.1', 'port': 9090, 'proto': 'http'},
'is_management': True,
},
'/ws': {
'backend': {'host': '127.0.0.1', 'port': 9091, 'proto': 'http'},
'is_websocket': True,
},
},
'auth_user': mgmt_user,
'auth_pass': mgmt_pass,
})
c.post(POST_NGINX_APPLY)
print(f' [proxy] Management proxy configured for {domain}')
except Exception as e:
print(f' [proxy] Warning: {e}', file=sys.stderr)
chown -R "$USER_DAEMON_NAME:$USER_GROUP" "${PROJECT_DIR}/config/auth"
chown -R "$USER_DAEMON_NAME:$USER_GROUP" "${PROJECT_DIR}/data/auth.db" 2>/dev/null || true
# Firewall config (interface detection done in bash above)
import json as _json
zones = {}
if wan_iface:
zones['public'] = {
'target': 'DEFAULT',
'interfaces': [i for i in wan_iface.split(',') if i],
'services': ['http', 'https', 'ssh'],
'masquerade': True,
# Helper: POST JSON to daemon API over Unix socket
_daemon_post() {
local endpoint="$1"
local json="$2"
local label="${3:-POST $endpoint}"
local resp
if resp=$(curl -s -f --unix-socket "$_SOCKET" \
"http://localhost${endpoint}" \
-H "Content-Type: application/json" \
-d "$json" 2>&1); then
log "$label"
return 0
else
warn "$label: $resp"
return 1
fi
}
if lan_ifaces:
zones['internal'] = {
'target': 'ACCEPT',
'interfaces': [i for i in lan_ifaces.split(',') if i],
'services': ['dhcp', 'dns', 'ntp'],
'masquerade': False,
}
# 1A. Self-signed certificate for management domain
_daemon_post "/acme/self-signed" "{\"domain\":\"$DOMAIN\"}" "Self-signed certificate"
# Always create vpn zone skeleton for later WireGuard setup
zones['vpn'] = {
'target': 'ACCEPT',
'interfaces': [],
'services': [],
'masquerade': False,
}
# 1C. Management proxy domain (no auth — JWT auth is handled by Flask)
mgmt_json="$(jq -n \
--arg domain "$DOMAIN" \
'{
domain: $domain,
backend: "webui",
cert: "selfsigned",
force_ssl: true
}')"
_daemon_post "/nginx/domains/update" "$mgmt_json" "Management domain configured" || \
_daemon_post "/nginx/domains/add" "$mgmt_json" "Management domain configured"
try:
c.post(POST_FIREWALL_CONFIG, {'zones': zones})
c.post(POST_FIREWALL_CONFIG_APPLY)
print(' [firewall] Zones configured and applied')
except Exception as e:
print(f' [firewall] Warning: {e}', file=sys.stderr)
_daemon_post "/nginx/apply" "{}" "Nginx config applied"
# IP forwarding
try:
c.post(POST_NETWORK_SYSCTL_SET, {'name': 'net.ipv4.ip_forward', 'value': '1'})
print(' [network] IP forwarding enabled')
except Exception as e:
print(f' [network] Warning: {e}', file=sys.stderr)
# Firewall zone assignment
if [[ -n "$WAN_IFACE" ]]; then
_daemon_post "/firewall/zones/interfaces" \
"$(jq -n --arg zone "public" --arg iface "$WAN_IFACE" \
'{zone: $zone, interfaces: [$iface]}')" \
"WAN interface assigned to public zone"
# Infer DHCP ranges (logged for user reference)
try:
ranges = c.get(GET_NETWORK_INFER_DHCP_RANGES)
for iface, rng in ranges.get('ranges', {}).items():
print(f' [suggestion] DHCP range for {iface}: {rng.get(\"start\")}-{rng.get(\"end\")}')
except Exception:
pass
"
log "Subsystem configuration complete"
# Open management services (HTTP, HTTPS, SSH) on the public/WAN zone
_daemon_post "/firewall/zones/services" \
"$(jq -n '{zone: "public", services: ["http", "https", "ssh"]}')" \
"Management services opened on public zone (http, https, ssh)"
fi
if [[ -n "$LAN_IFACES" ]]; then
# Convert comma-separated list to JSON array
LAN_JSON=$(echo "$LAN_IFACES" | tr ',' '\n' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | jq -R . | jq -s '.')
_daemon_post "/firewall/zones/interfaces" \
"$(jq -n --arg zone "internal" --argjson ifaces "$LAN_JSON" \
'{zone: $zone, interfaces: $ifaces}')" \
"LAN interfaces assigned to internal zone"
fi
unset _daemon_post
fi
systemctl start vacuum-wall >/dev/null 2>&1 && log "Started vacuum-wall WebUI" || warn "Could not start vacuum-wall WebUI"
@@ -472,7 +498,7 @@ echo -e " ${GREEN}Vacuum Wall installed successfully!${NC}"
echo "============================================"
echo ""
echo " Management UI: https://$DOMAIN"
echo " User: $MGMT_USER"
echo " Admin user: $MGMT_USER"
echo " Daemon service: vacuum-walld.service"
echo " WebUI service: vacuum-wall.service"
echo " ACME renewal: vacuum-wall-acme.timer"
@@ -496,7 +522,3 @@ echo " 3. Configure DHCP ranges for your LAN"
echo " 4. Add proxy domains with ACME certificates"
echo " 5. Set up WireGuard tunnel (optional)"
echo ""
echo " NOTE: A self-signed certificate was generated."
echo " From the WebUI, issue a real certificate for $DOMAIN"
echo " when DNS points to this appliance."
echo ""
@@ -1,5 +1,10 @@
#!/bin/bash
if [[ $EUID -ne 0 ]]; then
exec sudo "$0" "$@"
exit 0
fi
echo "systemctl restart nginx"
systemctl restart nginx
sleep 1
@@ -22,3 +27,5 @@ if [ "$failed" -eq 1 ]; then
exit 1
fi
exit 0
+62 -6
View File
@@ -1,27 +1,83 @@
#!/usr/bin/env bash
# Download and vendor libraries into vendor/.
# Run from the project root after updating the VERSION variables below.
# Files are named {pkg}-{ver}.{ext}, with {pkg}.{ext} symlinks for stable references.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
VENDOR="$PROJECT_DIR/vendor"
WEBUI_VENDOR="$PROJECT_DIR/webui/static/vendor"
# ---- Library versions ----
ACME_VERSION="3.1.3"
HTM_VERSION="3.1.1"
VENDOR="vendor"
# ---- Helpers ----
download() {
local name="$1" url="$2" dest="$3"
if [[ -n "${SKIP_DOWNLOAD:-}" ]]; then
echo "[skip] $name (SKIP_DOWNLOAD is set)"
echo "[skip] $name"
return
fi
if [[ -f "$dest" ]]; then
echo "[skip] $name (already present)"
return
fi
echo "[download] $name$dest"
curl -sfL -o "$dest" "$url"
}
ensure_symlink() {
local link="$1" target="$2"
if [[ -L "$link" ]]; then
cur=$(readlink "$link")
if [[ "$cur" != "$target" ]]; then
echo "[symlink] $link$target (updated)"
ln -sf "$target" "$link"
fi
elif [[ -e "$link" ]]; then
echo "[symlink] $link (replacing existing file)"
mv -f "$link" "${link}.bak" && ln -sf "$target" "$link"
else
echo "[symlink] $link$target"
ln -sf "$target" "$link"
fi
}
# ---- acme.sh ----
ACME_VERSIONED="$VENDOR/acme-${ACME_VERSION}.sh"
ACME_SYMLINK="$VENDOR/acme.sh"
download "acme.sh@${ACME_VERSION}" \
"https://raw.githubusercontent.com/acmesh-official/acme.sh/${ACME_VERSION}/acme.sh" \
"${VENDOR}/acme.sh"
"$ACME_VERSIONED"
chmod +x "$ACME_VERSIONED"
ensure_symlink "$ACME_SYMLINK" "acme-${ACME_VERSION}.sh"
chmod +x "${VENDOR}/acme.sh"
# ---- htm ----
HTM_VERSIONED="$VENDOR/htm-${HTM_VERSION}.js"
HTM_SYMLINK="$VENDOR/htm.js"
download "htm@${HTM_VERSION}" \
"https://cdn.jsdelivr.net/npm/htm@${HTM_VERSION}/mini/index.module.js" \
"$HTM_VERSIONED"
ensure_symlink "$HTM_SYMLINK" "htm-${HTM_VERSION}.js"
# ---- ACME_HOME (acme.sh runtime home) ----
ACME_HOME="${ACME_HOME:-$PROJECT_DIR/data/acme}"
mkdir -p "$ACME_HOME"
if [[ ! -x "$ACME_HOME/acme.sh" ]]; then
cp "$ACME_SYMLINK" "$ACME_HOME/acme.sh"
chmod +x "$ACME_HOME/acme.sh"
fi
# ---- Webui symlinks (point to versioned files) ----
mkdir -p "$WEBUI_VENDOR"
WEBUI_LINKS=(
"htm.js:../../../vendor/htm-${HTM_VERSION}.js"
)
for entry in "${WEBUI_LINKS[@]}"; do
IFS=':' read -r name target <<< "$entry"
ensure_symlink "${WEBUI_VENDOR}/${name}" "$target"
done
echo "[done] All libraries vendored."
+12 -2
View File
@@ -33,8 +33,8 @@ server {
ssl_certificate_key {{ certs_dir }}/{{ domain }}.key;
{% endif %}
{% elif has_management %}
ssl_certificate {{ acme_cert_dir }}/fullchain.cer;
ssl_certificate_key {{ acme_cert_dir }}/{{ domain }}.key;
ssl_certificate {{ certs_dir }}/{{ domain }}.crt;
ssl_certificate_key {{ certs_dir }}/{{ domain }}.key;
{% endif %}
include snippets/vacuum-wall-ssl.conf;
@@ -53,6 +53,16 @@ server {
{% endif %}
{% for ppath, pcfg in paths.items() %}
{% if pcfg.is_management %}
# SPA static assets — served from disk, no Flask round-trip.
# no-cache: browsers revalidate every load; unchanged files are 304s.
location /static/ {
alias {{ static_root }}/;
add_header Cache-Control "no-cache" always;
add_header X-Content-Type-Options nosniff always;
add_header Content-Security-Policy "default-src 'none'" always;
}
{% endif %}
{% if pcfg.is_websocket %}
# {{ ppath }} -> {{ pcfg.backend.host }}:{{ pcfg.backend.port }} (WebSocket)
location {{ ppath }} {
+16 -9
View File
@@ -9,43 +9,50 @@ Defaults:{{ USER_DAEMON_NAME }} secure_path="/usr/local/sbin:/usr/local/bin:/usr
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/sbin/nginx -t
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/sbin/nginx -s reload
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/systemctl is-active nginx
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp * /etc/nginx/*
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp * /etc/nginx/conf.d/*
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp * /etc/nginx/snippets/*
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp -- /run/vacuum-wall/include.tmp /etc/nginx/conf.d/vacuum-wall.conf
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp -- /run/vacuum-wall/ssl-snippet.tmp /etc/nginx/snippets/vacuum-wall-ssl.conf
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/rm /etc/nginx/conf.d/vacuum-wall.conf
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/rm /etc/nginx/snippets/vacuum-wall-ssl.conf
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/chown root\:root /etc/nginx/conf.d/vacuum-wall.conf
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/chown root\:root /etc/nginx/snippets/vacuum-wall-ssl.conf
# Dnsmasq management
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/systemctl reload dnsmasq
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/systemctl restart dnsmasq
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/systemctl is-active dnsmasq
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cat /var/lib/dnsmasq/dnsmasq.leases
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/tee /etc/dnsmasq.d/vacuum-wall.conf
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cat /var/lib/misc/dnsmasq.leases
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/mkdir -p /etc/dnsmasq.d
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp * /etc/dnsmasq.d/*
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp -- /run/vacuum-wall/dnsmasq.tmp /etc/dnsmasq.d/vacuum-wall.conf
# WireGuard management
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/wg-quick *
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/wg *
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp * /etc/wireguard/*
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp -- /run/vacuum-wall/wg0.conf.tmp /etc/wireguard/wg0.conf
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/chown root\:root /etc/wireguard/wg0.conf
# Network interface queries
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/sbin/ip -o link show
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/sbin/ip -o addr show
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/sbin/ip -o addr show *
# Networkd management
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /bin/networkctl status *
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /bin/networkctl reload
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /bin/networkctl reconfigure *
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp * /etc/systemd/network/*
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cp -- /run/vacuum-wall/99-*.network /etc/systemd/network/
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/rm /etc/systemd/network/*.network
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/mkdir -p /etc/systemd/network
# Sysctl
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/sbin/sysctl -w *
# ACME home permissions (acme.sh chmods its tree to owner-only modes:
# 700 on the config home, 600 on keys/confs — group access must be
# reopened so the shared two-user model can read the tree). Files only:
# the setgid directories (2775) already grant group rwx, and chmodding
# them would trip the daemon unit's RestrictSUIDSGID seccomp filter.
# The trailing * spans the file argument list.
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/chmod g+rwX {{ ACME_HOME }}/*
# Misc
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/journalctl --unit=* -n *
{{ USER_DAEMON_NAME }} ALL=(root) NOPASSWD: /usr/bin/cat /var/log/nginx/*
+9 -2
View File
@@ -3,8 +3,15 @@ Description=Vacuum Wall ACME Certificate Renewal
[Service]
Type=oneshot
User={{ USER_NAME }}
# Run as the daemon user, not the WebUI user: it owns the project tree
# (and the ACME home) in production, and acme.sh chmods its config home
# to 700 and its keys/confs to 600 on every run. Running as the WebUI
# user left the tree unreadable to the daemon (and vice versa) whenever
# the two users' runs interleaved.
User={{ USER_DAEMON_NAME }}
WorkingDirectory={{ PROJECT_DIR }}
Environment=ACME_HOME={{ PROJECT_DIR }}/data/acme
Environment=HOME={{ PROJECT_DIR }}
ExecStart={{ ACME_HOME }}/acme.sh --cron --home {{ ACME_HOME }} --config-home {{ ACME_HOME }}
# --log: persistent on-disk transcript of the raw CA exchange (journald
# captures stdout regardless; the file survives journal retention).
ExecStart={{ ACME_HOME }}/acme.sh --cron --home {{ ACME_HOME }} --config-home {{ ACME_HOME }} --log
+5 -1
View File
@@ -2,8 +2,12 @@
Description=Vacuum Wall ACME Certificate Renewal Timer
[Timer]
# Daily only: ZeroSSL backs off a failed validation for 24h per domain
# (Retry-After: 86400). With two runs a day every attempt landed inside
# the previous attempt's backoff window, re-arming it — a permanent
# renewal lockout. Attempts >24h apart are required for the backoff to
# ever expire (acme.sh discussion #6419).
OnCalendar=*-*-* 00:00:00
OnCalendar=*-*-* 12:00:00
Persistent=true
RandomizedDelaySec=300
+2 -1
View File
@@ -22,7 +22,8 @@ Environment=HOME={{ PROJECT_DIR }}
# Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ReadWritePaths={{ PROJECT_DIR }} {{ PROJECT_DIR }}/config {{ PROJECT_DIR }}/data /tmp
ReadWritePaths={{ PROJECT_DIR }} {{ PROJECT_DIR }}/config {{ PROJECT_DIR }}/data /tmp /var/log/vacuum-wall
LogsDirectory=vacuum-wall
PrivateTmp=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
+38 -1
View File
@@ -12,14 +12,51 @@ WorkingDirectory={{ PROJECT_DIR }}
ExecStart={{ PROJECT_DIR }}/.venv/bin/python -m daemon
Restart=on-failure
RestartSec=5
TimeoutStopSec=15
Environment=PATH=/usr/local/bin:/usr/bin
Environment=PYTHONUNBUFFERED=1
Environment=ACME_HOME={{ PROJECT_DIR }}/data/acme
# acme.sh routes its _info/_err lines through logger(1) -> journald when
# SYS_LOG is set (default: off). This journals manual issue/renew runs
# in real time under this unit, whose subprocess stdout is otherwise
# captured by the daemon and never seen by the journal.
# Levels: 3=error, 6=info, 7=debug.
Environment=SYS_LOG=6
Environment=HOME={{ PROJECT_DIR }}
# Runtime directories created before namespace setup. ProtectSystem=strict
# makes the whole hierarchy read-only, and namespace setup fails
# (exit 226/NAMESPACE) if any ReadWritePaths= entry is missing at spawn.
# /run is a fresh tmpfs at every boot, so volatile /run paths must be
# created up front (RuntimeDirectory= here; /run/firewalld via
# system/tmpfiles.d/vacuum-wall.conf and by firewalld itself) rather than
# at first use.
# vacuum-wall : secure temp files used during config apply
# nginx : /run/nginx (listed in ReadWritePaths)
RuntimeDirectory=vacuum-wall nginx
RuntimeDirectoryMode=0750
LogsDirectory=vacuum-wall
# Security hardening
ProtectSystem=strict
ReadWritePaths={{ PROJECT_DIR }} /tmp /etc/systemd/network /etc/nginx /etc/dnsmasq.d /etc/wireguard /run/sudo /run/firewalld /run/nginx /run/nginx.pid /var/log/nginx
# NOTE: every ReadWritePaths= entry must exist when the unit spawns or namespace
# setup fails (226/NAMESPACE). Volatile /run entries are pre-created:
# /run/vacuum-wall, /run/nginx → RuntimeDirectory= (above)
# /run/firewalld → system/tmpfiles.d/vacuum-wall.conf (and is
# present while firewalld runs, which starts
# before this unit)
# /run/sudo is intentionally NOT listed: the daemon's sudo children use the
# NOPASSWD whitelist and never need sudo's session directory (verified with
# the directory absent). Listing it made the unit crash-loop whenever it
# restarted after the last sudo session had ended and sudo removed /run/sudo.
# /run/nginx.pid IS listed: nginx -t opens the pid file for *writing* in
# addition to -s/acme reading it, so a read-only mount makes every daemon-side
# `nginx -t` (and therefore /nginx/apply) fail with EROFS. The file is
# pre-created by system/tmpfiles.d/vacuum-wall.conf so the ReadWritePaths=
# entry always exists at spawn (nginx rewrites it on start; nginx -t does
# not modify its contents).
ReadWritePaths={{ PROJECT_DIR }} /tmp /etc/systemd/network /etc/nginx /etc/dnsmasq.d /etc/wireguard /run/vacuum-wall /run/firewalld /run/nginx /run/nginx.pid /var/log/nginx /var/log/vacuum-wall
PrivateTmp=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
+25
View File
@@ -0,0 +1,25 @@
# Volatile /run entries that must exist before vacuum-walld spawns.
#
# vacuum-walld runs with ProtectSystem=strict and lists these paths in
# ReadWritePaths=; if a ReadWritePaths= entry is missing at spawn time,
# systemd's mount-namespace setup fails (exit 226/NAMESPACE) and the unit
# crash-loops without ever creating data/daemon.sock. /run is a fresh tmpfs
# at every boot, so every /run path the unit references needs a boot-time
# creator. Status per path:
#
# /run/vacuum-wall, /run/nginx -> unit RuntimeDirectory= (daemon-owned)
# /run/firewalld -> this file (the firewalld unit only creates
# it while firewalld itself is running)
# /run/nginx.pid -> this file (nginx rewrites it on start; the
# daemon's `nginx -t` must be able to open
# it for writing inside its ProtectSystem=strict
# namespace, so it needs both a boot-time
# creator and a ReadWritePaths= entry)
# /run/sudo -> not referenced by the unit (see
# ReadWritePaths note in vacuum-walld.service);
# the sudo package ships its own tmpfiles spec
#
# Applied at early boot by systemd-tmpfiles-setup.service and by the install
# script (`systemd-tmpfiles --create`) for existing hosts.
d /run/firewalld 0750 root root -
f /run/nginx.pid 0644 root root -
+272
View File
@@ -0,0 +1,272 @@
/**
* Tests for hoover/components/applyconfirm.js
*
* Component-level tests: VNode structure, buildRows logic,
* and integration behaviour. Run with `node tests/test-applyconfirm.js`.
*/
import { buildRows, isPending, SUBSYSTEM_LIST, applyResultToasts } from '../webui/static/hoover/components/applyconfirm.js';
const SUBSYSTEM_KEYS = SUBSYSTEM_LIST.map(s => s.key);
let passed = 0;
let failed = 0;
function test(name, fn) {
try {
fn();
console.log(` \u2713 ${name}`);
passed++;
} catch (e) {
console.error(` \u2717 ${name}: ${e.message}`);
failed++;
}
}
function assert(cond, msg) {
if (!cond) throw new Error(msg || 'Assertion failed');
}
function assertEq(a, b, msg) {
if (a !== b) throw new Error(msg || `Expected ${b}, got ${a}`);
}
function assertIncludes(str, substr, msg) {
if (!str.includes(substr)) throw new Error(msg || `Expected "${str}" to contain "${substr}"`);
}
console.log('Testing ApplyConfirm component\n');
// === isPending ===
test('isPending returns true for needs_apply', () => {
assertEq(isPending({ needs_apply: true }), true);
});
test('isPending returns true for pending_changes', () => {
assertEq(isPending({ pending_changes: true }), true);
});
test('isPending returns false when neither flag set', () => {
assertEq(isPending({}), false);
});
test('isPending returns false for explicit false', () => {
assertEq(isPending({ needs_apply: false, pending_changes: false }), false);
});
// === SUBSYSTEM_LIST ===
test('SUBSYSTEM_LIST contains 5 subsystems', () => {
assertEq(SUBSYSTEM_LIST.length, 5);
});
test('SUBSYSTEM_LIST uses networkd key (not network)', () => {
assertIncludes(SUBSYSTEM_KEYS.join(','), 'networkd', 'SUBSYSTEM_LIST should contain networkd');
assert(SUBSYSTEM_KEYS.indexOf('network') === -1, 'SUBSYSTEM_LIST should NOT contain network');
});
test('SUBSYSTEM_LIST keys match daemon response keys', () => {
const expectedKeys = ['firewall', 'dnsmasq', 'nginx', 'wireguard', 'networkd'];
for (const key of expectedKeys) {
assert(SUBSYSTEM_KEYS.includes(key), `SUBSYSTEM_LIST should contain ${key}`);
}
});
// === buildRows ===
test('buildRows returns 5 rows for empty subsystems', () => {
const rows = buildRows({}, {});
assertEq(rows.length, 5, 'should have 5 subsystem rows for empty state');
});
test('buildRows marks pending firewall subsystem correctly', () => {
const data = {
firewall: {
needs_apply: true,
changes: [
{ summary: 'Zone internal: interfaces changed', detail: '' },
],
},
};
const rows = buildRows(data, {});
const fwRow = rows[0];
assertIncludes(fwRow.props.class, 'pending', 'firewall row should have pending class');
});
test('buildRows changes are VNodes with proper structure', () => {
const data = {
firewall: {
needs_apply: true,
changes: [
{ summary: 'Zone internal: interfaces changed', detail: '' },
],
},
};
const rows = buildRows(data, {});
const fwRow = rows[0];
// Status span should contain "1 pending changes"
const statusSpan = fwRow.ch.find(c => c.props.class && c.props.class.includes('apply-subsystem-status'));
assert(statusSpan, 'should have status span');
const textChild = statusSpan.ch.find(c => c.tag === '#text');
assert(textChild && textChild.text.includes('pending changes'), 'status should contain pending changes count');
});
test('buildRows marks pending dnsmasq subsystem correctly', () => {
const data = {
dnsmasq: {
pending_changes: true,
changes: [
{ summary: 'DHCP/DNS configuration has unapplied changes', detail: '' },
],
},
};
const rows = buildRows(data, {});
const dnsmasqRow = rows[1];
assertIncludes(dnsmasqRow.props.class, 'pending', 'dnsmasq row should have pending class');
});
test('buildRows shows correct change count text', () => {
const data = {
dnsmasq: {
pending_changes: true,
changes: [
{ summary: 'Range 1', detail: '' },
{ summary: 'Range 2', detail: '' },
],
},
};
const rows = buildRows(data, {});
const dnsmasqRow = rows[1];
const statusSpan = dnsmasqRow.ch.find(c => c.props.class && c.props.class.includes('apply-subsystem-status'));
const textChild = statusSpan.ch.find(c => c.tag === '#text');
assertEq(textChild.text, '2 pending changes');
});
test('buildRows shows up-to-date for non-pending', () => {
const data = {
nginx: { pending_changes: false, changes: [] },
wireguard: { pending_changes: false, changes: [] },
};
const rows = buildRows(data, {});
const nginxRow = rows[2];
assert(
!nginxRow.props.class.includes('pending'),
'nginx row should not have pending class',
);
const statusSpan = nginxRow.ch.find(c => c.props.class && c.props.class.includes('apply-subsystem-status'));
const textChild = statusSpan.ch.find(c => c.tag === '#text');
assertEq(textChild.text, 'Up to date');
});
test('buildRows shows expand icon and details when expanded', () => {
const data = {
firewall: {
needs_apply: true,
changes: [
{ summary: 'Zone internal: interfaces changed', detail: '' },
{ summary: 'Zone dmz: services changed', detail: '' },
],
},
};
const rows = buildRows(data, { firewall: true });
assertEq(rows.length, 6, 'should have 6 items (5 rows + 1 detail section)');
});
test('buildRows hides expand icon when not expanded', () => {
const data = {
firewall: {
needs_apply: true,
changes: [
{ summary: 'Zone internal: interfaces changed', detail: '' },
],
},
};
const rows = buildRows(data, {});
assertEq(rows.length, 5, 'should only have 5 rows, no detail section');
});
test('buildRows pending flag but no changes treated as up-to-date', () => {
const data = {
firewall: { needs_apply: true, changes: [] },
};
const rows = buildRows(data, {});
const fwRow = rows[0];
assert(
!fwRow.props.class.includes('pending'),
'no changes = up to date',
);
});
test('buildRows detail section contains item VNodes', () => {
const data = {
firewall: {
needs_apply: true,
changes: [
{ summary: 'Zone internal: interfaces changed', detail: '' },
],
},
};
const rows = buildRows(data, { firewall: true });
const detailSection = rows[1];
assertIncludes(detailSection.props.class, 'apply-detail-section', 'should be detail section');
assert(detailSection.ch.length > 0, 'detail section should have children');
});
test('buildRows handles networkd key correctly', () => {
const data = {
networkd: {
pending_changes: true,
changes: [
{ summary: 'Network configuration has unapplied changes', detail: '' },
],
},
};
const rows = buildRows(data, {});
const networkdRow = rows[4]; // networkd is 5th in list
assertIncludes(networkdRow.props.class, 'pending', 'networkd row should have pending class');
});
test('buildRows row VNodes have correct tag', () => {
const data = {
firewall: { needs_apply: true, changes: [{ summary: 'test', detail: '' }] },
};
const rows = buildRows(data, {});
for (const row of rows.slice(0, 5)) {
assertEq(row.tag, 'div', 'row should be a div');
assert(row.props.class.includes('apply-subsystem-row'), 'row should have apply-subsystem-row class');
}
});
// === applyResultToasts ===
// apply-all returns 200 with { applied, errors } even when subsystems
// failed — resp.ok alone is not a success signal; errors must win.
test('applyResultToasts: errors suppress the success toast', () => {
const t = applyResultToasts({ applied: ['Network'], errors: { Firewall: 'refused' } }, 'All changes applied');
assertEq(t.success, null, 'no success toast when errors exist');
assertIncludes(t.error, 'Firewall — refused');
});
test('applyResultToasts: success toast when applied and no errors', () => {
const t = applyResultToasts({ applied: ['Firewall', 'Nginx'], errors: {} }, 'All changes applied');
assertEq(t.error, null);
assertEq(t.success, 'All changes applied');
});
test('applyResultToasts: no toast when nothing applied and no errors', () => {
const t = applyResultToasts({ applied: [], errors: {} }, 'All changes applied');
assertEq(t.error, null);
assertEq(t.success, null);
});
test('applyResultToasts: multiple errors are joined', () => {
const t = applyResultToasts({ applied: [], errors: { Firewall: 'a', Nginx: 'b' } }, 'ok');
assertIncludes(t.error, 'Firewall — a');
assertIncludes(t.error, 'Nginx — b');
});
test('applyResultToasts: null payload is safe', () => {
const t = applyResultToasts(null, 'ok');
assertEq(t.error, null);
assertEq(t.success, null);
});
console.log(`\n${passed} passed, ${failed} failed`);
process.exit(failed > 0 ? 1 : 0);
+315
View File
@@ -0,0 +1,315 @@
/**
* Tests for hoover/auth_model.js
*
* Model-level tests: the 'check' action (session validation, including the
* single refresh fallback on a 401) and the 'refresh' action (rotation).
*
* auth_model.js only pulls in model.js reactivity.js (no DOM at import),
* so the tests run under plain node with stubbed globals (fetch,
* sessionStorage, document, window, timers).
*
* Run with `node tests/test-auth-model.js`.
*/
import { createAuthModel } from '../webui/static/hoover/auth_model.js';
import { modelRegister, modelFetch, getModel } from '../webui/static/hoover/model.js';
let passed = 0;
let failed = 0;
const tests = [];
function test(name, fn) {
tests.push({ name, fn });
}
function assert(cond, msg) {
if (!cond) throw new Error(msg || 'Assertion failed');
}
function assertEq(a, b, msg) {
if (a !== b) throw new Error((msg || 'Assertion failed') + `: got ${JSON.stringify(a)}, want ${JSON.stringify(b)}`);
}
/* ── Stubs ─────────────────────────────────────────────────── */
function makeStorage(initial = {}) {
const m = new Map(Object.entries(initial));
return {
getItem: (k) => (m.has(k) ? m.get(k) : null),
setItem: (k, v) => m.set(k, String(v)),
removeItem: (k) => m.delete(k),
keys: () => [...m.keys()],
};
}
/** Programmed URL → response queue. Missing routes are 500. */
function makeFetch() {
const byUrl = new Map();
globalThis.fetch = async (url) => {
const arr = byUrl.get(url) || [];
const r = arr.length ? arr.shift() : { status: 500, body: { ok: false, error: 'unprogrammed route' } };
return {
ok: r.status >= 200 && r.status < 300,
status: r.status,
json: async () => r.body,
};
};
const state = { calls: [], route: (url, status, body) => { if (!byUrl.has(url)) byUrl.set(url, []); state.calls.push(url); byUrl.get(url).push({ status, body }); } };
return state;
}
/** Stub the TTL refresh timer so the node process never waits on it. */
const _timers = [];
globalThis.setTimeout = (fn, ms) => { _timers.push({ fn, ms }); return _timers.length; };
globalThis.clearTimeout = (id) => { if (id && _timers[id - 1]) _timers[id - 1] = undefined; };
const storedSession = {
'vw:access': 'access-old',
'vw:refresh': 'refresh-old',
'vw:session_id': 'sess-old',
'vw:access_ttl': '900000',
};
/**
* Stubs + model registration for one scenario.
* @param {{initialStorage?: object, hash?: string}} [cfg]
* @returns {{calls: string[], route: function, events: Array}}
*/
function setup({ initialStorage = storedSession, hash = '#/dashboard' } = {}) {
const { calls, route } = makeFetch();
globalThis.sessionStorage = makeStorage(initialStorage);
globalThis.document = { location: { hash } };
const events = [];
globalThis.window = {
dispatchEvent: (e) => events.push(e),
addEventListener: () => {},
};
modelRegister('auth', createAuthModel());
return { calls, route, events };
}
/** Run the auth model action through the real modelFetch (dedup, hooks). */
async function act(action) {
await modelFetch('auth', { action });
}
/* ── Tests ─────────────────────────────────────────────────── */
test('check 200 returns verified identity on the stored tokens', async () => {
const s = setup();
s.route('/api/auth/session', 200, {
ok: true,
data: { user: { username: 'alice' }, permissions: { firewall: 'rw' } },
});
await act('check');
assertEq(s.calls.length, 1, 'session check only');
assertEq(s.calls[0], '/api/auth/session');
const data = getModel('auth').data;
assertEq(data.token, 'access-old', 'stored access token kept');
assertEq(data.refresh, 'refresh-old', 'stored refresh token kept');
assertEq(data.session_id, 'sess-old', 'stored session_id kept');
assertEq(data.user?.username, 'alice', 'verified user merged');
assert(data.permissions?.firewall === 'rw', 'verified permissions merged');
});
test('check 401 with stored refresh token: one refresh, session preserved', async () => {
const s = setup();
s.route('/api/auth/session', 401, { ok: false, error: 'unauthorized' });
s.route('/api/auth/refresh', 200, {
ok: true,
data: {
tokens: { access_token: 'access-new', refresh_token: 'refresh-new', session_id: 'sess-new' },
user: { username: 'alice' },
permissions: { firewall: 'rw' },
access_ttl: 300,
},
});
await act('check');
assertEq(s.calls.length, 2, 'exactly one refresh attempt');
assertEq(s.calls[0], '/api/auth/session', 'session check first');
assertEq(s.calls[1], '/api/auth/refresh', 'refresh fallback second');
const data = getModel('auth').data;
assertEq(data.token, 'access-new', 'rotated access token');
assertEq(data.refresh, 'refresh-new', 'rotated refresh token');
assertEq(data.session_id, 'sess-new', 'rotated session_id binding wins');
assertEq(data.user?.username, 'alice', 'refreshed identity');
assertEq(globalThis.sessionStorage.getItem('vw:session_id'), 'sess-new', 'storage carries rotated session_id');
assert(s.events.length === 0, 'no terminal event on recovered session');
});
test('check 401 without refresh token: terminal, no refresh attempted', async () => {
const s = setup({ initialStorage: { 'vw:access': 'access-old', 'vw:session_id': 'sess-old' } });
s.route('/api/auth/session', 401, { ok: false, error: 'unauthorized' });
await act('check');
assertEq(s.calls.length, 1, 'session check only');
assertEq(s.calls[0], '/api/auth/session');
const data = getModel('auth').data;
assert(!data?.token, 'terminal: no token');
assert(globalThis.sessionStorage.keys().length === 0, 'storage cleared');
assertEq(globalThis.document.location.hash, '/login', 'redirected to login');
assert(s.events.some((e) => e.type === 'auth:logout'), 'terminal auth:logout dispatched');
});
test('check 401 with failed refresh: terminal', async () => {
const s = setup();
s.route('/api/auth/session', 401, { ok: false, error: 'unauthorized' });
s.route('/api/auth/refresh', 401, { ok: false, error: 'invalid refresh token' });
await act('check');
assertEq(s.calls.length, 2, 'refresh was attempted');
assert(!getModel('auth').data?.token, 'terminal: no token');
assert(globalThis.sessionStorage.keys().length === 0, 'storage cleared');
assertEq(globalThis.document.location.hash, '/login', 'redirected to login');
assert(s.events.some((e) => e.type === 'auth:logout'), 'terminal auth:logout dispatched');
});
test('refresh action rotates tokens; new session_id wins, omitted fields fall back', async () => {
const s = setup();
s.route('/api/auth/refresh', 200, {
ok: true,
data: {
tokens: { access_token: 'a2', refresh_token: 'r2', session_id: 's2' },
access_ttl: 300,
user: { username: 'alice' },
permissions: { firewall: 'rw' },
},
});
await act('refresh');
assertEq(s.calls.length, 1, 'single refresh call');
assertEq(s.calls[0], '/api/auth/refresh');
const data = getModel('auth').data;
assertEq(data.token, 'a2', 'rotated access token');
assertEq(data.refresh, 'r2', 'rotated refresh token');
assertEq(data.session_id, 's2', 'new session_id wins');
assertEq(data.ttl, 300 * 1000, 'ttl from access_ttl seconds → ms');
assertEq(globalThis.sessionStorage.getItem('vw:session_id'), 's2', 'storage rotated');
assertEq(data.user?.username, 'alice', 'user from response');
});
/* ── exp-claim TTL tests ───────────────────────────────────── */
/** Base64url-encode a JSON object (JWT segment builder). */
function b64url(obj) {
return btoa(JSON.stringify(obj))
.replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '');
}
/** Build a structurally valid (unsigned) JWT whose exp is offsetSeconds from now. */
function makeJwt(offsetSeconds) {
return [
b64url({ alg: 'HS256' }),
b64url({
sub: 'alice',
exp: Math.floor(Date.now() / 1000) + offsetSeconds,
iat: Math.floor(Date.now() / 1000),
type: 'access',
session_id: 'sess-jwt',
}),
b64url({ sig: true }),
].join('.');
}
/** Most recent defined entry in the captured timer queue. */
function lastTimer() {
for (let i = _timers.length - 1; i >= 0; i--) if (_timers[i]) return _timers[i];
return null;
}
test('check 200: ttl is the token\'s remaining lifetime (exp claim), not the stored full TTL', async () => {
const s = setup({
initialStorage: {
'vw:access': makeJwt(600), // expires in 10 min…
'vw:refresh': 'refresh-old',
'vw:session_id': 'sess-jwt',
'vw:access_ttl': '900000', // …but the stored full TTL says 15 min
},
});
s.route('/api/auth/session', 200, {
ok: true,
data: { user: { username: 'alice' }, permissions: { firewall: 'rw' } },
});
await act('check');
const ttl = getModel('auth').data.ttl;
assert(ttl > 590 * 1000 && ttl <= 600 * 1000,
`remaining ttl (~600s), not the stored 900s: got ${ttl}`);
// scheduleRefresh fires at ttl - 60s — the timer must target the real expiry.
const t = lastTimer();
assert(t && t.ms > 530 * 1000 && t.ms <= 540 * 1000,
`refresh timer targets expiry - 60s: got ${t && t.ms}`);
});
test('check 200: already-expired token falls back to the stored TTL (401 recovery path applies)', async () => {
const s = setup({
initialStorage: {
'vw:access': makeJwt(-10), // already expired
'vw:refresh': 'refresh-old',
'vw:session_id': 'sess-jwt',
'vw:access_ttl': '900000',
},
});
s.route('/api/auth/session', 200, {
ok: true,
data: { user: { username: 'alice' }, permissions: {} },
});
await act('check');
assertEq(getModel('auth').data.ttl, 900 * 1000, 'fallback to stored ttl');
});
test('check 200: non-JWT stored token falls back to the stored TTL', async () => {
const s = setup(); // default storage carries the non-JWT 'access-old'
s.route('/api/auth/session', 200, {
ok: true,
data: { user: { username: 'alice' }, permissions: {} },
});
await act('check');
assertEq(getModel('auth').data.ttl, 900 * 1000, 'fallback to stored ttl');
});
test('refresh action: rotated ttl comes from the new token\'s exp claim', async () => {
const s = setup();
s.route('/api/auth/refresh', 200, {
ok: true,
data: {
tokens: { access_token: makeJwt(450), refresh_token: 'r2', session_id: 's2' },
access_ttl: 300, // full TTL — must lose to the exp claim
user: { username: 'alice' },
permissions: { firewall: 'rw' },
},
});
await act('refresh');
const ttl = getModel('auth').data.ttl;
assert(ttl > 440 * 1000 && ttl <= 450 * 1000,
`exp-based ttl (~450s), not access_ttl 300s: got ${ttl}`);
});
test('login action: ttl comes from the issued token\'s exp claim', async () => {
const s = setup();
await modelFetch('auth', {
action: 'login',
payload: {
tokens: { access_token: makeJwt(900), refresh_token: 'r1', session_id: 's1' },
access_ttl: 900,
user: { username: 'alice' },
permissions: { firewall: 'rw' },
},
});
const ttl = getModel('auth').data.ttl;
assert(ttl > 890 * 1000 && ttl <= 900 * 1000,
`exp-based ttl (~900s): got ${ttl}`);
});
/* ── Runner ────────────────────────────────────────────────── */
(async () => {
for (const { name, fn } of tests) {
try {
await fn();
console.log(` \u2713 ${name}`);
passed++;
} catch (e) {
console.error(` \u2717 ${name}: ${e.message}`);
failed++;
}
}
console.log(`${passed + failed} tests: ${passed} passed, ${failed} failed`);
process.exitCode = failed ? 1 : 0;
})();
+87
View File
@@ -0,0 +1,87 @@
/**
* Tests for hoover/components/applyconfirm.js CancelConfirm
*
* Component-level tests: VNode structure of the cancel button.
* Run with `node tests/test-cancelconfirm.js`.
*/
import { CancelConfirm, buildRows, SUBSYSTEM_LIST } from '../webui/static/hoover/components/applyconfirm.js';
let passed = 0;
let failed = 0;
function test(name, fn) {
try {
fn();
console.log(` \u2713 ${name}`);
passed++;
} catch (e) {
console.error(` \u2717 ${name}: ${e.message}`);
failed++;
}
}
function assert(cond, msg) {
if (!cond) throw new Error(msg || 'Assertion failed');
}
function assertEq(a, b, msg) {
if (a !== b) throw new Error(msg || `Expected ${b}, got ${a}`);
}
function assertIncludes(str, substr, msg) {
if (!str.includes(substr)) throw new Error(msg || `Expected "${str}" to contain "${substr}"`);
}
console.log('Testing CancelConfirm component\n');
test('CancelConfirm renders a button vnode', () => {
const vnode = CancelConfirm();
assertEq(vnode.tag, 'button');
});
test('CancelConfirm default label', () => {
const vnode = CancelConfirm();
const text = vnode.ch.find(c => c.tag === '#text');
assert(text, 'should have text child');
assertEq(text.text, 'Cancel All Changes');
});
test('CancelConfirm default class is danger', () => {
const vnode = CancelConfirm();
assertIncludes(vnode.props.class, 'btn-danger');
assertIncludes(vnode.props.class, 'btn');
});
test('CancelConfirm accepts a custom class', () => {
const vnode = CancelConfirm({ cls: 'btn btn-sm btn-danger' });
assertEq(vnode.props.class, 'btn btn-sm btn-danger');
});
test('CancelConfirm accepts a custom label', () => {
const vnode = CancelConfirm({ label: 'Discard Changes' });
const text = vnode.ch.find(c => c.tag === '#text');
assertEq(text.text, 'Discard Changes');
});
test('CancelConfirm has a click handler', () => {
const vnode = CancelConfirm();
assert(typeof vnode.props['on:click'] === 'function', 'on:click should be a function');
});
// === shared modal row builder (used by the cancel modal) ===
test('buildRows still drives the cancel modal rows', () => {
const data = {
firewall: { needs_apply: true, changes: [{ summary: 'Zone internal: interfaces changed', detail: '' }] },
dnsmasq: { pending_changes: true, changes: [{ summary: 'x', detail: '' }] },
};
const rows = buildRows(data, {});
assertEq(rows.length, SUBSYSTEM_LIST.length);
const fwRow = rows[0];
assertIncludes(fwRow.props.class, 'pending');
const dmRow = rows[1];
assertIncludes(dmRow.props.class, 'pending');
});
console.log(`\n${passed} passed, ${failed} failed`);
process.exit(failed > 0 ? 1 : 0);
+270
View File
@@ -0,0 +1,270 @@
/**
* Tests for hoover/dirty.js pending-edit marker matching.
*
* dirty.js has no imports DOM-free at import, so the tests run under
* plain node (same pattern as test-model-set.js).
*
* Run with `node tests/test-dirty.js`.
*/
import { dirtySet, isDirty, dirtyTitle, dirtyInfo, orphanInfo, fwDirty, fwIsDirty, fwTitle, fwInfo } from '../webui/static/hoover/dirty.js';
let passed = 0;
let failed = 0;
const tests = [];
function test(name, fn) {
tests.push({ name, fn });
}
function assert(cond, msg) {
if (!cond) throw new Error(msg || 'Assertion failed');
}
function assertEq(a, b, msg) {
if (a !== b) throw new Error((msg || 'Assertion failed') + `: got ${JSON.stringify(a)}, want ${JSON.stringify(b)}`);
}
/* ── dirtySet ────────────────────────────────────────────────── */
test('dirtySet collects pending paths from pending_diff', () => {
const set = dirtySet({ pending_diff: [
{ path: 'dhcp.ranges[0].start', action: 'changed' },
{ path: 'dns.domain', action: 'added' },
]});
assert(set.has('dhcp.ranges[0].start'), 'first path collected');
assert(set.has('dns.domain'), 'second path collected');
assertEq(set.size, 2, 'exactly two paths');
});
test('dirtySet skips diff entries without a path', () => {
const set = dirtySet({ pending_diff: [null, {}, { action: 'changed' }, { path: 'a.b' }] });
assertEq(set.size, 1, 'only well-formed entries');
assert(set.has('a.b'), 'valid path collected');
});
test('dirtySet is empty when pending_diff is absent', () => {
assertEq(dirtySet(null).size, 0, 'null status');
assertEq(dirtySet({}).size, 0, 'empty status');
assertEq(dirtySet({ pending_diff: 'nope' }).size, 0, 'non-array pending_diff');
});
/* ── never-applied sentinel ──────────────────────────────────── */
test('dirtySet marks everything dirty when saved but never applied', () => {
const set = dirtySet({ pending_changes: true, pending_diff: [] });
assertEq(set.size, 1, 'sentinel only');
assert(isDirty(set, 'dhcp.ranges[0].start'), 'any path is dirty');
assert(isDirty(set, 'interface.listen_port'), 'any other path is dirty');
assertEq(dirtyTitle(set, 'dhcp.ranges[0].start'), 'Configuration saved but not applied yet', 'sentinel tooltip');
});
test('dirtySet has no sentinel when there is no pending state', () => {
const set = dirtySet({ pending_changes: false, pending_diff: [] });
assert(!isDirty(set, 'dhcp.ranges'), 'clean when nothing is pending');
assertEq(dirtyTitle(set, 'dhcp.ranges'), '', 'no tooltip when clean');
});
test('dirtySet has no sentinel when a real diff exists', () => {
const set = dirtySet({
pending_changes: true,
pending_diff: [{ path: 'dns.domain', action: 'changed' }],
});
assert(isDirty(set, 'dns.domain'), 'matching path is dirty');
assert(!isDirty(set, 'dhcp.ranges'), 'unrelated path stays clean');
assertEq(dirtyTitle(set, 'dns.domain'), 'Unapplied changes: dns.domain', 'normal tooltip, not the sentinel');
});
/* ── line matching ───────────────────────────────────────────── */
test('isDirty matches an exact pending leaf', () => {
const set = dirtySet({ pending_diff: [{ path: 'interface.listen_port', action: 'changed' }] });
assert(isDirty(set, 'interface.listen_port'), 'equal path is dirty');
});
test('a pending list marks every indexed row (ancestor of element)', () => {
const set = dirtySet({ pending_diff: [{ path: 'dhcp.ranges', action: 'changed' }] });
for (const i of [0, 1, 12]) {
assert(isDirty(set, `dhcp.ranges[${i}]`), `row ${i} is dirty`);
assert(isDirty(set, `dhcp.ranges[${i}].start`), `row ${i} field is dirty`);
}
});
test('a pending row field marks the list (descendant of element)', () => {
const set = dirtySet({ pending_diff: [{ path: 'dhcp.ranges[0].start', action: 'changed' }] });
assert(isDirty(set, 'dhcp.ranges'), 'the list container is dirty');
assert(isDirty(set, 'dhcp'), 'the top-level container is dirty');
});
test('unrelated paths do not match', () => {
const set = dirtySet({ pending_diff: [{ path: 'dns.domain', action: 'changed' }] });
assert(!isDirty(set, 'dhcp.ranges'), 'different root');
});
test('index brackets do not prefix-match across digits', () => {
const set = dirtySet({ pending_diff: [{ path: 'dhcp.ranges[1]', action: 'changed' }] });
assert(!isDirty(set, 'dhcp.ranges[12]'), 'ranges[1] must not mark row 12');
assert(!isDirty(set, 'dhcp.ranges[10]'), 'ranges[1] must not mark row 10');
assert(isDirty(set, 'dhcp.ranges[1]'), 'the exact row is dirty');
});
test('plain keys do not prefix-match similar names', () => {
const set = dirtySet({ pending_diff: [{ path: 'interface.listen_port', action: 'changed' }] });
assert(!isDirty(set, 'interfaces.eth0'), 'interface must not mark interfaces.eth0');
assert(!isDirty(set, 'interface2.port'), 'interface must not mark interface2');
});
test('isDirty is false for an empty or missing set', () => {
assert(!isDirty(new Set(), 'a.b'), 'empty set');
assert(!isDirty(null, 'a.b'), 'null set');
assert(!isDirty(dirtySet({}), 'a.b'), 'status with no pending');
});
test('isDirty tolerates an empty path', () => {
const set = dirtySet({ pending_diff: [{ path: 'a.b', action: 'changed' }] });
assert(!isDirty(set, ''), 'empty element path is not dirty');
assert(!isDirty(set, null), 'null element path is not dirty');
});
/* ── dirtyTitle / dirtyInfo ──────────────────────────────────── */
test('dirtyTitle lists all matching pending paths sorted', () => {
const set = dirtySet({ pending_diff: [
{ path: 'dhcp.ranges[1].start', action: 'changed' },
{ path: 'dhcp.ranges[0].end', action: 'changed' },
{ path: 'dns.domain', action: 'changed' },
]});
assertEq(
dirtyTitle(set, 'dhcp.ranges'),
'Unapplied changes: dhcp.ranges[0].end, dhcp.ranges[1].start',
'both rows listed, sorted, unrelated path excluded',
);
});
test('dirtyTitle is empty when the element is clean', () => {
const set = dirtySet({ pending_diff: [{ path: 'dns.domain', action: 'changed' }] });
assertEq(dirtyTitle(set, 'dhcp.ranges'), '', 'no tooltip for unrelated element');
});
test('dirtyInfo returns the full marker object', () => {
const set = dirtySet({ pending_diff: [{ path: 'dns.domain', action: 'changed' }] });
const hit = dirtyInfo(set, 'dns.domain');
assertEq(hit.dirty, true, 'dirty flag');
assertEq(hit.class, 'config-dirty', 'class');
assertEq(hit.title, 'Unapplied changes: dns.domain', 'tooltip');
const miss = dirtyInfo(set, 'dhcp.ranges');
assertEq(miss.dirty, false, 'clean flag');
assertEq(miss.class, '', 'clean class');
assertEq(miss.title, '', 'clean title');
});
/* ── orphanInfo (removed dict keys) ──────────────────────────── */
test('orphanInfo flags a removed peer with no live row', () => {
const set = dirtySet({ pending_diff: [{ path: 'peers.p1', action: 'removed' }] });
const info = orphanInfo(set, 'peers', ['peers.p2', 'peers.p3']);
assertEq(info.dirty, true, 'orphan is dirty');
assertEq(info.class, 'config-dirty', 'orphan class');
assertEq(info.title, 'Unapplied changes: peers.p1', 'orphan tooltip');
});
test('orphanInfo is clean when the pending path still has a live row', () => {
const set = dirtySet({ pending_diff: [{ path: 'peers.p1.endpoint', action: 'changed' }] });
assertEq(orphanInfo(set, 'peers', ['peers.p1', 'peers.p2']).dirty, false, 'matched child is not an orphan');
});
test('orphanInfo flags a removed peer when no peers remain', () => {
const set = dirtySet({ pending_diff: [{ path: 'peers.p1', action: 'removed' }] });
assertEq(orphanInfo(set, 'peers', []).dirty, true, 'no children means the orphan stands');
});
test('orphanInfo ignores pending paths outside the root', () => {
const set = dirtySet({ pending_diff: [{ path: 'interface.listen_port', action: 'changed' }] });
assertEq(orphanInfo(set, 'peers', ['peers.p1']).dirty, false, 'unrelated root');
});
test('orphanInfo is clean when the root itself is pending', () => {
// A whole-dict `peers` change marks every child row instead; the
// container-level marker would be redundant.
const set = dirtySet({ pending_diff: [{ path: 'peers', action: 'changed' }] });
assertEq(orphanInfo(set, 'peers', ['peers.p1']).dirty, false, 'root-pending is not an orphan');
assert(isDirty(set, 'peers.p1'), 'but the rows are still marked');
});
test('orphanInfo is clean for an empty set or the never-applied sentinel', () => {
assertEq(orphanInfo(new Set(), 'peers', []).dirty, false, 'empty set');
const sentinel = dirtySet({ pending_changes: true, pending_diff: [] });
assertEq(orphanInfo(sentinel, 'peers', []).dirty, false, 'sentinel: element markers already cover it');
});
test('orphanInfo lists multiple orphans sorted', () => {
const set = dirtySet({ pending_diff: [
{ path: 'peers.b', action: 'removed' },
{ path: 'peers.a', action: 'removed' },
{ path: 'peers.c.field', action: 'changed' },
]});
const info = orphanInfo(set, 'peers', ['peers.c']);
assertEq(info.title, 'Unapplied changes: peers.a, peers.b', 'only the orphans, sorted');
});
/* ── firewall zone + type granularity ────────────────────────── */
test('fwDirty builds a zone-to-types map', () => {
const m = fwDirty({
pending: [
{ zone: 'public', type: 'services' },
{ zone: 'public', type: 'rich_rules' },
{ zone: 'dmz', type: 'interfaces' },
{ zone: null },
{ zone: 'lan' },
],
});
assertEq(m.size, 3, 'three zones (null-zone entry skipped, typeless zone kept)');
assert(m.get('public').has('services'), 'public services');
assert(m.get('public').has('rich_rules'), 'public rich_rules');
assert(m.get('dmz').has('interfaces'), 'dmz interfaces');
assert(m.get('lan').size === 0, 'typeless zone has an empty type set');
});
test('fwIsDirty by zone and by zone+type', () => {
const m = fwDirty({ pending: [{ zone: 'public', type: 'services' }] });
assert(fwIsDirty(m, 'public'), 'zone-only match');
assert(fwIsDirty(m, 'public', 'services'), 'zone+type match');
assert(!fwIsDirty(m, 'public', 'rich_rules'), 'wrong type');
assert(!fwIsDirty(m, 'dmz'), 'unknown zone');
assert(!fwIsDirty(new Map(), 'public'), 'empty map');
});
test('fwInfo and fwTitle carry the pending types', () => {
const m = fwDirty({
pending: [
{ zone: 'public', type: 'rich_rules' },
{ zone: 'public', type: 'services' },
],
});
const zone = fwInfo(m, 'public');
assertEq(zone.dirty, true, 'zone dirty');
assertEq(zone.class, 'config-dirty', 'zone class');
assertEq(zone.title, 'Unapplied changes: rich_rules, services', 'zone tooltip lists all types');
const typed = fwInfo(m, 'public', 'services');
assertEq(typed.title, 'Unapplied changes: services', 'typed tooltip lists only that type');
assertEq(fwInfo(m, 'dmz').dirty, false, 'unknown zone clean');
assertEq(fwTitle(m, 'nope'), '', 'no tooltip for unknown zone');
});
/* ── Runner ──────────────────────────────────────────────────── */
(async () => {
for (const { name, fn } of tests) {
try {
await fn();
console.log(` \u2713 ${name}`);
passed++;
} catch (e) {
console.error(` \u2717 ${name}: ${e.message}`);
failed++;
}
}
console.log(`${passed + failed} tests: ${passed} passed, ${failed} failed`);
process.exitCode = failed ? 1 : 0;
})();
+111
View File
@@ -0,0 +1,111 @@
/**
* Tests for hoover/model.js modelSet() the WS data-streaming entry point.
*
* modelSet() bypasses the fetch cycle: it assigns directly to the reactive
* model, clears loading unconditionally (schema defaults mean model.data is
* never null), clears error, and never sets refreshing.
*
* model.js imports only reactivity.js DOM-free at import, so the tests
* run under plain node (same pattern as test-auth-model.js).
*
* Run with `node tests/test-model-set.js`.
*/
import { modelRegister, modelFetch, getModel, modelSet } from '../webui/static/hoover/model.js';
import { SUBSYSTEMS } from '../webui/static/hoover/schema.js';
let passed = 0;
let failed = 0;
const tests = [];
function test(name, fn) {
tests.push({ name, fn });
}
function assert(cond, msg) {
if (!cond) throw new Error(msg || 'Assertion failed');
}
function assertEq(a, b, msg) {
if (a !== b) throw new Error((msg || 'Assertion failed') + `: got ${JSON.stringify(a)}, want ${JSON.stringify(b)}`);
}
/** Deep equality for objects/arrays (assertEq is reference-based). */
function assertDeep(a, b, msg) {
if (JSON.stringify(a) !== JSON.stringify(b)) throw new Error((msg || 'Assertion failed') + `: got ${JSON.stringify(a)}, want ${JSON.stringify(b)}`);
}
/* ── Tests ─────────────────────────────────────────────────── */
test('modelSet assigns data and clears loading unconditionally', () => {
modelRegister('firewall', {
subsystem: 'firewall',
defaultData: SUBSYSTEMS.firewall.defaults,
fetch: async () => ({}),
});
const m = getModel('firewall');
assertEq(m.loading, true, 'registering model is loading');
assertEq(m.data, SUBSYSTEMS.firewall.defaults, 'schema defaults pre-populated');
assertEq(m.refreshing, false, 'not refreshing at rest');
modelSet('firewall', { zones: { public: {} }, pending: { pending: [] } });
assertEq(m.loading, false, 'real data ends the initial load');
assertEq(m.refreshing, false, 'modelSet never sets refreshing');
assertEq(m.error, null, 'modelSet clears error');
assertDeep(m.data.zones?.public, {}, 'data assigned to the reactive model');
});
test('modelSet is a no-op for an unregistered name', () => {
assertEq(typeof modelSet('no-such-model', { x: 1 }), 'undefined', 'no throw, no return');
});
test('modelSet clears a fetch-set error and replaces error-state data', async () => {
modelRegister('dnsmasq', {
subsystem: 'dnsmasq',
defaultData: SUBSYSTEMS.dnsmasq.defaults,
fetch: async () => { throw new Error('boom'); },
});
const m = getModel('dnsmasq');
await modelFetch('dnsmasq');
assertEq(m.error, 'boom', 'fetch failure sets error');
assertEq(m.loading, false, 'fetch failure clears loading');
assertEq(m.data, SUBSYSTEMS.dnsmasq.defaults, 'failed fetch keeps schema defaults');
const delta = { leases: [{ mac: 'aa:bb', ip: '10.0.0.9' }] };
modelSet('dnsmasq', delta);
assertEq(m.error, null, 'subsequent real data clears the error');
assertEq(m.data.leases?.[0]?.mac, 'aa:bb', 'delta replaces default data');
});
test('modelSet works repeatedly without flag corruption', () => {
modelRegister('system', {
subsystem: 'system',
defaultData: SUBSYSTEMS.system.defaults,
fetch: async () => ({}),
});
const m = getModel('system');
modelSet('system', { load: { load1: 0.1 } });
modelSet('system', { load: { load1: 0.2 } });
assertEq(m.loading, false, 'still not loading');
assertEq(m.refreshing, false, 'still not refreshing');
assertEq(m.error, null, 'no error');
assertEq(m.data.load?.load1, 0.2, 'latest delta wins');
});
/* ── Runner ────────────────────────────────────────────────── */
(async () => {
for (const { name, fn } of tests) {
try {
await fn();
console.log(` \u2713 ${name}`);
passed++;
} catch (e) {
console.error(` \u2717 ${name}: ${e.message}`);
failed++;
}
}
console.log(`${passed + failed} tests: ${passed} passed, ${failed} failed`);
process.exitCode = failed ? 1 : 0;
})();
+175
View File
@@ -0,0 +1,175 @@
/**
* Tests for the 3-second HTTP-fallback contract (Phase 3e).
*
* app.js cannot be imported under node (it imports every page and touches
* the DOM), so this covers the model-layer contract the deferred fetch
* decides on:
*
* - a model registered with schema defaults keeps `loading: true`
* (data is never null that's why the guard is `if (model.loading)`)
* - `modelSet()` or a completed `modelFetch()` clears `loading`
* - the `if (model.loading) modelFetch(name)` decision fires ONLY for
* still-loading models a WS-delivered snapshot suppresses the HTTP
* fallback for that model
*
* Uses a fake setTimeout queue and recording fetch stubs (no timers, no
* network) same pattern as test-auth-model.js.
*
* Run with `node tests/test-reconnect-fallback.js`.
*/
import { modelRegister, modelFetch, getModel, modelSet } from '../webui/static/hoover/model.js';
import { SUBSYSTEMS } from '../webui/static/hoover/schema.js';
let passed = 0;
let failed = 0;
const tests = [];
function test(name, fn) {
tests.push({ name, fn });
}
function assert(cond, msg) {
if (!cond) throw new Error(msg || 'Assertion failed');
}
function assertEq(a, b, msg) {
if (a !== b) throw new Error((msg || 'Assertion failed') + `: got ${JSON.stringify(a)}, want ${JSON.stringify(b)}`);
}
/** Deep equality for objects/arrays (assertEq is reference-based). */
function assertDeep(a, b, msg) {
if (JSON.stringify(a) !== JSON.stringify(b)) throw new Error((msg || 'Assertion failed') + `: got ${JSON.stringify(a)}, want ${JSON.stringify(b)}`);
}
/* ── Stubs ─────────────────────────────────────────────────── */
/** Fake setTimeout queue — captures scheduled fallbacks, never runs them. */
const _timers = [];
globalThis.setTimeout = (fn, ms) => { _timers.push({ fn, ms }); return _timers.length; };
globalThis.clearTimeout = (id) => { if (id && _timers[id - 1]) _timers[id - 1] = undefined; };
/** Drain pending timers in order, like a real 3s elapse. */
async function runTimers() {
while (_timers.length) {
const t = _timers.shift();
if (t) await t.fn();
}
}
/** Register one state-backed model with recording fetch. */
function registerModel(name) {
const calls = { count: 0 };
modelRegister(name, {
subsystem: name,
defaultData: SUBSYSTEMS[name].defaults,
fetch: async () => {
calls.count++;
return { fetched: true, name };
},
});
return calls;
}
/* ── Mirrors app.js fetchInitialData() decision logic ───────── */
/** Schedule the per-model 3s fallback timers (as app.js does). */
function scheduleFallbacks(names) {
for (const name of names) {
setTimeout(() => {
const model = getModel(name);
if (model.loading) {
modelFetch(name);
}
}, 3000);
}
}
/* ── Tests ─────────────────────────────────────────────────── */
test('schema-defaulted model: loading stays true until modelSet or fetch completes', () => {
registerModel('firewall');
const m = getModel('firewall');
assertEq(m.loading, true, 'fresh model is loading');
assertEq(m.data, SUBSYSTEMS.firewall.defaults, 'data is schema defaults (never null)');
assertEq(m.error, null, 'no error yet');
modelSet('firewall', { zones: { public: {} } });
assertEq(m.loading, false, 'modelSet clears loading');
registerModel('dnsmasq');
const d = getModel('dnsmasq');
assertEq(d.loading, true, 'a different fresh model is still loading');
});
test('3s fallback fetches only models still loading (snapshot suppresses HTTP)', async () => {
const firewallCalls = registerModel('firewall');
const dnsmasqCalls = registerModel('dnsmasq');
scheduleFallbacks(['firewall', 'dnsmasq']);
// The WS snapshot arrived first for firewall only.
modelSet('firewall', { zones: { internal: {} } });
await runTimers();
assertEq(firewallCalls.count, 0, 'snapshot-delivered model: no HTTP fallback');
assertEq(dnsmasqCalls.count, 1, 'still-loading model: HTTP fallback fired');
const fw = getModel('firewall');
assertEq(fw.loading, false, 'firewall not loading');
assertDeep(fw.data.zones?.internal, {}, 'firewall keeps the snapshot data, not fetch output');
const dm = getModel('dnsmasq');
assertEq(dm.loading, false, 'completed fetch clears loading');
assertEq(dm.data.fetched, true, 'dnsmasq got the fallback data');
});
test('a re-fired decision never double-fetches a settled model', async () => {
const calls = registerModel('acme');
scheduleFallbacks(['acme']);
await runTimers();
assertEq(calls.count, 1, 'first fallback fetch');
// A later decision pass (e.g. reconnect path) must not re-fetch.
const model = getModel('acme');
if (model.loading) modelFetch('acme');
await runTimers();
assertEq(calls.count, 1, 'no double fetch once settled');
});
test('fallback fetch failure lands in model.error, self-heals on next data', async () => {
modelRegister('wireguard', {
subsystem: 'wireguard',
defaultData: SUBSYSTEMS.wireguard.defaults,
fetch: async () => { throw new Error('state not populated yet'); },
});
const m = getModel('wireguard');
setTimeout(() => { if (m.loading) modelFetch('wireguard'); }, 3000);
await runTimers();
assertEq(m.error, 'state not populated yet', 'failure sets model.error');
assertEq(m.loading, false, 'failure still clears loading (finally)');
assertEq(m.data, SUBSYSTEMS.wireguard.defaults, 'schema defaults preserved on failure');
modelSet('wireguard', { up: false });
assertEq(m.error, null, 'next real data clears the error');
assertEq(m.data.up, false, 'real data lands');
});
/* ── Runner ────────────────────────────────────────────────── */
(async () => {
for (const { name, fn } of tests) {
try {
await fn();
console.log(` \u2713 ${name}`);
passed++;
} catch (e) {
console.error(` \u2717 ${name}: ${e.message}`);
failed++;
}
}
console.log(`${passed + failed} tests: ${passed} passed, ${failed} failed`);
process.exitCode = failed ? 1 : 0;
})();
+301
View File
@@ -0,0 +1,301 @@
/**
* Tests for hoover/render.js component lifecycle (per-container #comp registry).
*
* Regression: the #comp lifecycle registry and expanded-content cache were
* module-globals, pruned per-container inside normalizeVNodesWithLifecycle().
* Because commitAll() commits #sidebar (no #comp) before #main (the page
* #comp), every sidebar commit unmounted+pruned the page from the global
* registry, so the following #main commit treated the page as newly mounted
* and re-ran load(). For pages whose load() re-mutates reactive state with
* fresh values each run (passkeys.js, users.js), every re-run scheduled
* another commit an infinite unmount/remount/load loop (~100 fetches/s),
* leaving the page stuck on "Loading...".
*
* render.js pulls in vdom.js + component.js DOM-only at commit time, so the
* tests run under plain node with a minimal fake DOM (same pattern as
* test-auth-model.js / test-model-set.js).
*
* Run with `node tests/test-render-lifecycle.js`
* (optional arg 1: hoover root, defaults to ../webui/static/hoover).
*
* NOTE: against buggy (global-registry) code the self-mutation test spins the
* infinite remount loop and saturates the event loop the process hangs
* instead of failing an assertion (mirrors the live symptom). Run under an
* external `timeout` when checking old checkouts:
* timeout 30 node tests/test-render-lifecycle.js <hoover-root>
*/
import { pathToFileURL } from 'node:url';
import path from 'node:path';
const HOOVER_ROOT = process.argv[2]
? pathToFileURL(path.resolve(process.argv[2])).href + '/'
: new URL('../webui/static/hoover/', import.meta.url).href;
/* ── Minimal fake DOM ───────────────────────────────────────── */
class FakeEl {
constructor(tag) {
this.tagName = String(tag || 'div').toUpperCase();
this.nodeType = 1;
this.childNodes = [];
this.parentNode = null;
this.style = { cssText: '' };
this.attributes = {};
this._listeners = {};
this.className = '';
this.value = '';
this.checked = false;
this.selected = false;
this.disabled = false;
}
get firstChild() { return this.childNodes[0] || null; }
setAttribute(k, v) { this.attributes[k] = String(v); }
removeAttribute(k) { delete this.attributes[k]; }
appendChild(c) {
if (c.parentNode) c.parentNode.removeChild(c);
c.parentNode = this;
this.childNodes.push(c);
return c;
}
insertBefore(c, ref) {
if (c.parentNode) c.parentNode.removeChild(c);
c.parentNode = this;
const i = ref ? this.childNodes.indexOf(ref) : this.childNodes.length;
this.childNodes.splice(i === -1 ? this.childNodes.length : i, 0, c);
return c;
}
removeChild(c) {
const i = this.childNodes.indexOf(c);
if (i !== -1) this.childNodes.splice(i, 1);
c.parentNode = null;
return c;
}
replaceChild(nd, od) {
const i = this.childNodes.indexOf(od);
if (i !== -1) this.childNodes[i] = nd;
od.parentNode = null;
nd.parentNode = this;
return od;
}
addEventListener(ev, fn) { (this._listeners[ev] ||= []).push(fn); }
removeEventListener(ev, fn) {
const arr = this._listeners[ev] || [];
const i = arr.indexOf(fn);
if (i !== -1) arr.splice(i, 1);
}
}
class FakeText {
constructor(text) { this.nodeType = 3; this.nodeValue = String(text); this.parentNode = null; }
}
globalThis.document = {
createElement: (tag) => new FakeEl(tag),
createTextNode: (t) => new FakeText(t),
};
globalThis.window = { addEventListener: () => {} };
/* ── Imports (dynamic: hoover root is injectable) ───────────── */
const { reactive } = await import(HOOVER_ROOT + 'reactivity.js');
const { h } = await import(HOOVER_ROOT + 'vdom.js');
const { render } = await import(HOOVER_ROOT + 'render.js');
const { definePage, hComp } = await import(HOOVER_ROOT + 'component.js');
let passed = 0;
let failed = 0;
const tests = [];
function test(name, fn) { tests.push({ name, fn }); }
function assert(cond, msg) { if (!cond) throw new Error(msg || 'Assertion failed'); }
function assertEq(a, b, msg) {
if (a !== b) throw new Error((msg || 'Assertion failed') + `: got ${a}, want ${b}`);
}
const flush = () => new Promise(r => setTimeout(r, 20));
/**
* Build a page whose load() mutates reactive state (like passkeys.js
* loadCredentials: refreshing=true before the fetch, credentials=<new array>
* and refreshing=false after fresh values on every run).
*/
function makePage(label, counters, title) {
const state = reactive({ loading: true, done: 0 });
return {
state,
page: definePage({
title: title || undefined,
init: () => state,
async load(s) {
counters.loads++;
counters.loadKeys.push(label);
s.done = (s.done || 0) + 1; // fresh value every run → schedules a commit
s.loading = false;
},
onUnmount: () => { counters.unmounts++; counters.unmountKeys.push(label); },
render: () => h('div', { class: 'card' }, `${label}-body`),
}),
};
}
const freshCounters = () => ({ loads: 0, unmounts: 0, loadKeys: [], unmountKeys: [] });
test('initial mount runs load() exactly once', async () => {
const c = freshCounters();
const { page } = makePage('A', c);
const sidebar = new FakeEl('div');
const main = new FakeEl('div');
render(sidebar, () => h('div', { class: 'sidebar' }, 'nav'));
render(main, () => hComp(page, '/page-a'));
await flush();
assertEq(c.loads, 1, 'load ran once');
assertEq(c.unmounts, 0, 'no unmounts');
});
test('external reactive update does NOT re-mount the page', async () => {
const c = freshCounters();
const { page } = makePage('A', c);
const sidebar = new FakeEl('div');
const main = new FakeEl('div');
render(sidebar, () => h('div', { class: 'sidebar' }, 'nav'));
render(main, () => hComp(page, '/page-a'));
await flush();
assertEq(c.loads, 1, 'baseline');
// Simulate a WS tick / toast / any reactive mutation outside the page.
const external = reactive({ n: 1 });
for (let i = 0; i < 3; i++) {
external.n += 1;
await flush();
}
assertEq(c.loads, 1, 'load still ran exactly once after 3 external updates');
assertEq(c.unmounts, 0, 'page was never unmounted');
});
test('page load() self-mutations do not re-trigger load (no infinite loop)', async () => {
const c = freshCounters();
const { page } = makePage('A', c);
const sidebar = new FakeEl('div');
const main = new FakeEl('div');
render(sidebar, () => h('div', { class: 'sidebar' }, 'nav'));
render(main, () => hComp(page, '/page-a'));
// load() mutates reactive state on every run — give the (buggy) loop time
// to spin. With the per-container registry it must stay at exactly one run.
await flush();
await flush();
await flush();
assertEq(c.loads, 1, 'no remount loop driven by load\'s own state mutations');
assertEq(c.unmounts, 0, 'no spurious unmounts');
});
test('navigation unmounts the old page once and mounts the new page once', async () => {
const c = freshCounters();
const a = makePage('A', c);
const b = makePage('B', c);
const nav = reactive({ path: '/page-a' });
const sidebar = new FakeEl('div');
const main = new FakeEl('div');
render(sidebar, () => h('div', { class: 'sidebar' }, 'nav'));
render(main, () => hComp(nav.path === '/page-a' ? a.page : b.page, nav.path));
await flush();
assertEq(c.loads, 1, 'A mounted');
nav.path = '/page-b';
await flush();
assertEq(c.loads, 2, 'B mounted once');
assertEq(c.unmounts, 1, 'A unmounted once');
assertEq(c.unmountKeys[0], 'A', 'A was the unmounted page');
// navigate back — A mounts again with preserved state (load re-runs by design)
nav.path = '/page-a';
await flush();
assertEq(c.loads, 3, 'A re-mounted after navigation back');
assertEq(c.unmounts, 2, 'B unmounted');
assertEq(a.state.done, 2, 'A state preserved across unmount (2 loads total)');
});
test('two #comp containers: updates in one root do not disturb the other', async () => {
const c = freshCounters();
const left = makePage('L', c);
const right = makePage('R', c);
const l = new FakeEl('div');
const r = new FakeEl('div');
render(l, () => hComp(left.page, '/left'));
render(r, () => hComp(right.page, '/right'));
await flush();
assertEq(c.loads, 2, 'both pages mounted');
const external = reactive({ n: 1 });
for (let i = 0; i < 3; i++) { external.n += 1; await flush(); }
assertEq(c.loads, 2, 'neither page re-mounted');
assertEq(c.unmounts, 0, 'neither page unmounted');
});
/* ── Tab title (definePage `title`) ─────────────────────────── */
test('mounting a titled page sets document.title', async () => {
const c = freshCounters();
const { page } = makePage('T', c, 'Titled - Vacuum Wall');
const main = new FakeEl('div');
document.title = 'base';
render(main, () => hComp(page, '/titled'));
await flush();
assertEq(document.title, 'Titled - Vacuum Wall', 'title applied on mount');
});
test('a page without a title leaves document.title untouched', async () => {
const c = freshCounters();
const { page } = makePage('U', c);
const main = new FakeEl('div');
document.title = 'unchanged';
render(main, () => hComp(page, '/untitled'));
await flush();
assertEq(document.title, 'unchanged', 'no title → document.title untouched');
});
test('navigation updates document.title; remount re-applies idempotently', async () => {
const c = freshCounters();
const a = makePage('A', c, 'Alpha - Vacuum Wall');
const b = makePage('B', c, 'Beta - Vacuum Wall');
const nav = reactive({ path: '/page-a' });
const main = new FakeEl('div');
render(main, () => hComp(nav.path === '/page-a' ? a.page : b.page, nav.path));
await flush();
assertEq(document.title, 'Alpha - Vacuum Wall', 'A title on first mount');
nav.path = '/page-b';
await flush();
assertEq(document.title, 'Beta - Vacuum Wall', 'B title after navigation');
nav.path = '/page-a';
await flush();
assertEq(document.title, 'Alpha - Vacuum Wall', 'A title re-applied on remount');
});
test('mounting an untitled page does not reset a previously set title', async () => {
const c = freshCounters();
const a = makePage('A', c, 'Alpha - Vacuum Wall');
const b = makePage('B', c);
const nav = reactive({ path: '/page-a' });
const main = new FakeEl('div');
render(main, () => hComp(nav.path === '/page-a' ? a.page : b.page, nav.path));
await flush();
assertEq(document.title, 'Alpha - Vacuum Wall', 'baseline');
nav.path = '/page-b';
await flush();
assertEq(document.title, 'Alpha - Vacuum Wall', 'untitled mount keeps prior title');
});
/* ── Runner ─────────────────────────────────────────────────── */
(async () => {
for (const { name, fn } of tests) {
try {
await fn();
console.log(` \u2713 ${name}`);
passed++;
} catch (e) {
console.error(` \u2717 ${name}: ${e.message}`);
failed++;
}
}
console.log(`${passed + failed} tests: ${passed} passed, ${failed} failed`);
process.exitCode = failed ? 1 : 0;
})();
+220
View File
@@ -0,0 +1,220 @@
/**
* Tests for hoover/websocket.js handleMessage() WS data streaming.
*
* handleMessage is driven through a real connect() against a stubbed
* globalThis.WebSocket: we record the constructed instance and call its
* onmessage handler with serialized daemonclient messages, then assert the
* reactive model state. Covers the snapshot fast path, per-subsystem deltas
* (including the networkdnetwork mapping), null-payload guards, and that
* retired/legacy message types are ignored (no model mutation, no throw).
*
* websocket.js pulls in model.js reactivity.js and auth_model.js
* (DOM-free at import), so it runs under plain node with stubbed globals.
*
* Run with `node tests/test-ws-handler.js`.
*/
import { connect } from '../webui/static/hoover/websocket.js';
import { modelRegister, getModel, modelSet } from '../webui/static/hoover/model.js';
import { SUBSYSTEMS } from '../webui/static/hoover/schema.js';
let passed = 0;
let failed = 0;
const tests = [];
function test(name, fn) {
tests.push({ name, fn });
}
function assert(cond, msg) {
if (!cond) throw new Error(msg || 'Assertion failed');
}
function assertEq(a, b, msg) {
if (a !== b) throw new Error((msg || 'Assertion failed') + `: got ${JSON.stringify(a)}, want ${JSON.stringify(b)}`);
}
/** Deep equality for objects/arrays (assertEq is reference-based). */
function assertDeep(a, b, msg) {
if (JSON.stringify(a) !== JSON.stringify(b)) throw new Error((msg || 'Assertion failed') + `: got ${JSON.stringify(a)}, want ${JSON.stringify(b)}`);
}
/* ── Stubs ─────────────────────────────────────────────────── */
function makeStorage(initial = {}) {
const m = new Map(Object.entries(initial));
return {
getItem: (k) => (m.has(k) ? m.get(k) : null),
setItem: (k, v) => m.set(k, String(v)),
removeItem: (k) => m.delete(k),
keys: () => [...m.keys()],
};
}
// Stub the TTL refresh timer so the node process never waits on it.
const _timers = [];
globalThis.setTimeout = (fn, ms) => { _timers.push({ fn, ms }); return _timers.length; };
globalThis.clearTimeout = (id) => { if (id && _timers[id - 1]) _timers[id - 1] = undefined; };
globalThis.location = { protocol: 'http:', host: '127.0.0.1:9090' };
globalThis.sessionStorage = makeStorage({ 'vw:access': 'tok-abc.def.ghi' });
globalThis.document = { location: { hash: '#/dashboard' } };
globalThis.window = { dispatchEvent: () => {}, addEventListener: () => {} };
/** Records each constructed WebSocket so tests can drive onmessage. */
class FakeWebSocket {
static instances = [];
constructor(url, protocols) {
this.url = url;
this.protocols = protocols;
this.readyState = 1;
this.onopen = null;
this.onclose = null;
this.onerror = null;
this.onmessage = null;
FakeWebSocket.instances.push(this);
}
close() { this.readyState = 3; }
send() {}
}
globalThis.WebSocket = FakeWebSocket;
/** Register the auth + state models the WS handler depends on. */
function setupModels() {
modelRegister('auth', { subsystem: 'auth', fetch: async () => ({}) });
modelSet('auth', { token: 'tok-abc.def.ghi', user: { username: 'admin' } });
for (const [name, subsystem] of [
['firewall', 'firewall'], ['dnsmasq', 'dnsmasq'], ['nginx', 'nginx'],
['acme', 'acme'], ['wireguard', 'wireguard'], ['network', 'networkd'], ['system', 'system'],
]) {
modelRegister(name, {
subsystem,
defaultData: SUBSYSTEMS[subsystem].defaults,
fetch: async () => ({}),
});
}
}
/** Establish a fresh WS connection and return the recorded instance. */
function freshConnect() {
const prev = FakeWebSocket.instances.at(-1);
if (prev && prev.readyState <= 1) { prev.onclose = null; prev.close(); }
connect();
return FakeWebSocket.instances.at(-1);
}
/** Feed one daemon→client message through the recorded instance. */
function emit(inst, msg) {
inst.onmessage({ data: JSON.stringify(msg) });
}
const SNAPSHOT = {
type: 'snapshot',
data: {
firewall: { zones: { public: {} } },
dnsmasq: { leases: [] },
nginx: null, // collector failed → must be skipped
acme: { certs: [] },
wireguard: { up: true },
networkd: { interfaces: { eth0: {} } },
system: { load: { load1: 0.5 } },
},
};
/* ── Tests ─────────────────────────────────────────────────── */
test('connect() sends the raw JWT as the Sec-WebSocket-Protocol subprotocol', () => {
setupModels();
const inst = freshConnect();
assert(inst, 'a WS instance was constructed');
assertEq(inst.url, 'ws://127.0.0.1:9090/ws', 'WS URL from origin');
assert(Array.isArray(inst.protocols), 'subprotocols passed');
assertEq(inst.protocols[0], 'tok-abc.def.ghi', 'bare JWT (no Bearer prefix)');
});
test('snapshot fast path sets every non-null model; null entries are skipped', () => {
setupModels();
const inst = freshConnect();
emit(inst, SNAPSHOT);
assertDeep(getModel('firewall').data.zones?.public, {}, 'firewall patched');
assertDeep(getModel('dnsmasq').data.leases, [], 'dnsmasq patched');
assertDeep(getModel('acme').data.certs, [], 'acme patched');
assertEq(getModel('wireguard').data.up, true, 'wireguard patched');
assertDeep(getModel('network').data.interfaces?.eth0, {}, 'networkd mapped → network');
assertEq(getModel('system').data.load?.load1, 0.5, 'system patched');
// nginx data was null — modelSet was skipped entirely.
const nginx = getModel('nginx');
assertDeep(nginx.data, SUBSYSTEMS.nginx.defaults, 'null entry keeps schema defaults');
assertEq(nginx.loading, true, 'null entry never clears loading');
// Non-null models had loading cleared by modelSet.
assertEq(getModel('firewall').loading, false, 'loading cleared on real data');
});
test('versions delta patches the mapped subsystem model', () => {
setupModels();
const inst = freshConnect();
const before = JSON.stringify(getModel('firewall').data);
emit(inst, { type: 'versions', subsystem: 'firewall', data: { zones: { dmz: {} } } });
assert(getModel('firewall').data.zones?.dmz !== undefined, 'firewall delta applied');
assertEq(JSON.stringify(getModel('dnsmasq').data), JSON.stringify(SUBSYSTEMS.dnsmasq.defaults), 'other models untouched');
});
test('networkd delta maps to the network model', () => {
setupModels();
const inst = freshConnect();
emit(inst, { type: 'versions', subsystem: 'networkd', data: { interfaces: { lo: {} } } });
assertDeep(getModel('network').data.interfaces?.lo, {}, 'networkd → network');
});
test('a null payload delta never overwrites good data (defense in depth)', () => {
setupModels();
const inst = freshConnect();
emit(inst, SNAPSHOT); // networkd has data
const kept = getModel('network').data;
emit(inst, { type: 'tick', subsystem: 'networkd', data: null });
assertDeep(getModel('network').data, kept, 'null data left model untouched');
emit(inst, { type: 'versions', subsystem: 'firewall', data: null });
assertDeep(getModel('firewall').data.zones?.public, {}, 'null firewall data left model untouched');
});
test('retired/legacy message types are ignored (no mutation, no throw)', () => {
setupModels();
const inst = freshConnect();
emit(inst, SNAPSHOT);
const before = {};
for (const n of ['firewall', 'dnsmasq', 'nginx', 'acme', 'wireguard', 'network', 'system']) {
before[n] = JSON.stringify(getModel(n).data) + '|' + getModel(n).loading;
}
const legacy = [
{ type: 'versions', updated: { firewall: 1 } }, // legacy dict form
{ type: 'tick', subsystems: ['firewall', 'wireguard'] }, // legacy array form
{ type: 'refresh', topic: 'firewall' },
{ type: 'notify', topic: 'firewall' },
{ type: 'status', topic: 'firewall' },
{ type: 'unknown' },
];
for (const msg of legacy) emit(inst, msg);
for (const n of Object.keys(before)) {
const now = JSON.stringify(getModel(n).data) + '|' + getModel(n).loading;
assertEq(now, before[n], `model ${n} unchanged by legacy message`);
}
});
/* ── Runner ────────────────────────────────────────────────── */
(async () => {
for (const { name, fn } of tests) {
try {
await fn();
console.log(` \u2713 ${name}`);
passed++;
} catch (e) {
console.error(` \u2717 ${name}: ${e.message}`);
failed++;
}
}
console.log(`${passed + failed} tests: ${passed} passed, ${failed} failed`);
process.exitCode = failed ? 1 : 0;
})();
+65
View File
@@ -58,6 +58,25 @@ class TestRunAcme:
assert cmd[0] == "/usr/local/bin/acme.sh"
assert "sudo" not in cmd
@patch("lib.acme._find_acme")
@patch("lib.acme.subprocess.run")
def test_log_flag_is_last(self, mock_run, mock_find):
# --log <file> must trail the subcommand args: acme.sh would
# otherwise consume the first subcommand arg as its file argument.
# The explicit file path (not a bare trailing --log) is required
# because a valueless trailing --log makes acme.sh's arg loop
# double-shift under dash and fail with "shift: can't shift that
# many".
mock_find.return_value = "/usr/local/bin/acme.sh"
mock_run.return_value = MagicMock(returncode=0, stdout="", stderr="")
acme._run_acme(["--issue", "-d", "example.com"])
cmd = mock_run.call_args[0][0]
assert cmd.count("--log") == 1
assert cmd[-2] == "--log"
assert cmd[-1].endswith("acme.sh.log")
assert cmd.index("--issue") < cmd.index("--log")
assert "example.com" in cmd
class TestParseListOutput:
def test_parses_single_entry(self):
@@ -117,6 +136,20 @@ class TestParseListOutput:
result = acme._parse_list_output(raw)
assert result == []
def test_pipe_separated_raw_format(self):
"""Pipe-separated output with empty fields (what --listraw produces)."""
raw = (
"Main_Domain|KeyLength|SAN_Domains|Profile|CA|Created|Renew\n"
'example.com|"ec-256"|no||ZeroSSL.com|2026-01-01|2026-07-01\n'
)
result = acme._parse_list_output(raw)
assert len(result) == 1
assert result[0]["main_domain"] == "example.com"
assert result[0]["profile"] == ""
assert result[0]["ca"] == "ZeroSSL.com"
assert result[0]["created"] == "2026-01-01"
assert result[0]["renew"] == "2026-07-01"
class TestDaysUntil:
def test_future_date(self):
@@ -244,3 +277,35 @@ class TestHasAutoRenew:
with patch.object(acme, "_ACME_HOME", acme_dir):
result = acme._has_auto_renew("nonexistent.com")
assert result is False
class TestSummarizeAcmeOutput:
def test_last_two_lines(self):
out = (
"[2026-09-04] line one\n[2026-09-04] retry failed\n[2026-09-04] giving up\n"
)
assert acme._summarize_acme_output(out) == "retry failed; giving up"
def test_strips_timestamps_and_log_pointer(self):
out = "[ts] work\nPlease check log file /x/acme.sh.log\n[ts] done\n"
assert acme._summarize_acme_output(out) == "work; done"
def test_empty_returns_placeholder(self):
assert acme._summarize_acme_output("") == "(no output)"
def test_preserves_permission_denied_outside_tail(self):
out = (
"[ts] starting\n"
"[ts] /data/acme/account.conf: Permission denied\n"
"[ts] step three\n"
"[ts] step four\n"
)
summary = acme._summarize_acme_output(out)
# The permission line is not among the final two, but the
# actionable-error matcher (daemon/collectors/acme.py) keys off it.
assert "account.conf: Permission denied" in summary
assert summary.count("; ") == 2 # capped at three lines
def test_permission_denied_in_tail_not_duplicated(self):
out = "[ts] ok\n[ts] account.conf: Permission denied\n"
assert acme._summarize_acme_output(out) == "ok; account.conf: Permission denied"
+187 -1
View File
@@ -40,6 +40,11 @@ def _ne(func, **kw):
return _patch(f"webui.api.network.{func}", **kw)
def _st(func, **kw):
"""Patch daemon.client.{func} in the status blueprint namespace."""
return _patch(f"webui.api.status.{func}", **kw)
@pytest.fixture
def client():
from flask import Flask
@@ -462,7 +467,7 @@ class TestProxyDomains:
mock_post.return_value = {"domain": "ex.com"}
resp = client.post(
"/api/proxy/domains",
json={"domain": "ex.com", "backend_host": "10.0.0.1", "backend_port": 80},
json={"domain": "ex.com", "backend": "webui"},
)
assert resp.status_code == 200
@@ -525,6 +530,64 @@ class TestCertsIssue:
assert resp.status_code == 400
class TestCertsRenew:
@_ce("post")
def test_start_renew(self, mock_post, client):
mock_post.return_value = {"request_id": "abc123", "domain": "example.com"}
resp = client.post("/api/certs/example.com/renew")
assert resp.status_code == 200
data = resp.get_json()
assert data["ok"] is True
assert data["data"]["request_id"] == "abc123"
args = mock_post.call_args.args
assert args[0] == ("POST", "/acme/renew")
assert args[1] == {"domain": "example.com"}
@_ce("post")
def test_start_renew_rejected(self, mock_post, client):
from daemon.client import BadRequest
mock_post.side_effect = BadRequest("'domain' is required")
resp = client.post("/api/certs/example.com/renew")
assert resp.status_code == 400
assert resp.get_json()["ok"] is False
@_ce("post")
def test_start_renew_runtime_error(self, mock_post, client):
mock_post.side_effect = RuntimeError("daemon unreachable")
resp = client.post("/api/certs/example.com/renew")
assert resp.status_code == 500
assert resp.get_json()["ok"] is False
class TestCertsRenewStatus:
@_ce("get")
def test_success(self, mock_get, client):
mock_get.return_value = {
"request_id": "abc123",
"domain": "example.com",
"status": "completed",
"steps": [],
}
resp = client.get("/api/certs/renew/abc123")
assert resp.status_code == 200
data = resp.get_json()
assert data["ok"] is True
assert data["data"]["status"] == "completed"
args = mock_get.call_args.args
assert args[0] == ("GET", "/acme/renew/status")
assert args[1] == {"id": "abc123"}
@_ce("get")
def test_not_found(self, mock_get, client):
from daemon.client import NotFound
mock_get.side_effect = NotFound("renewal request not found")
resp = client.get("/api/certs/renew/unknown")
assert resp.status_code == 404
assert resp.get_json()["ok"] is False
class TestCertsEmail:
def test_missing_email(self, client):
resp = client.post("/api/certs/email", json={})
@@ -883,3 +946,126 @@ class TestNetworkApplyAll:
mock_post.side_effect = RuntimeError("apply failed")
resp = client.post("/api/network/apply")
assert resp.status_code == 500
# ============================================================================
# Status
# ============================================================================
@pytest.fixture
def status_client():
from flask import Flask
from webui.api.status import bp as status_bp
app = Flask(__name__)
app.register_blueprint(status_bp, url_prefix="/api/status")
return app.test_client()
class TestStatusPending:
@_st("get")
def test_advisory_fields_passthrough(self, mock_get, status_client):
from daemon.iface import GET_STATUS_PENDING
mock_get.return_value = {
"firewall": {
"needs_apply": False,
"change_count": 0,
"changes": [],
"uncovered_interfaces": ["eth1"],
"coverage_warnings": [
"Interfaces not in any firewall zone: eth1 — clients "
"on those segments lose connectivity and DHCP"
],
},
"dnsmasq": {
"pending_changes": False,
"summary": "Up to date",
"changes": [],
},
"nginx": {"pending_changes": False, "summary": "Up to date", "changes": []},
"wireguard": {
"pending_changes": False,
"summary": "Up to date",
"changes": [],
},
"networkd": {
"pending_changes": False,
"summary": "Up to date",
"changes": [],
},
"total_changes": 0,
}
resp = status_client.get("/api/status/pending")
assert resp.status_code == 200
data = resp.get_json()
assert data["ok"] is True
assert data["data"]["firewall"]["uncovered_interfaces"] == ["eth1"]
assert data["data"]["firewall"]["coverage_warnings"]
assert data["data"]["total_changes"] == 0
mock_get.assert_called_once_with(GET_STATUS_PENDING)
@_st("get")
def test_runtime_error(self, mock_get, status_client):
mock_get.side_effect = RuntimeError("no daemon")
resp = status_client.get("/api/status/pending")
assert resp.status_code == 500
assert resp.get_json()["ok"] is False
class TestStatusRefresh:
def test_filtered_subsystems_passed_through(self, status_client):
"""The subsystem body is forwarded to the daemon POST endpoint."""
from daemon.iface import POST_STATUS_REFRESH
with _st("post") as mock_post:
mock_post.return_value = {"firewall": {"zones": {}}}
resp = status_client.post(
"/api/status/refresh", json={"subsystems": ["firewall"]}
)
assert resp.status_code == 200
data = resp.get_json()
assert data["ok"] is True
assert data["data"] == {"firewall": {"zones": {}}}
mock_post.assert_called_once_with(
POST_STATUS_REFRESH, {"subsystems": ["firewall"]}
)
def test_empty_body_forwards_empty_dict(self, status_client):
"""An empty body becomes {} (daemon-side 'all subsystems' default)."""
from daemon.iface import POST_STATUS_REFRESH
with _st("post") as mock_post:
mock_post.return_value = {}
resp = status_client.post("/api/status/refresh")
assert resp.status_code == 200
mock_post.assert_called_once_with(POST_STATUS_REFRESH, {})
@_st("post")
def test_runtime_error(self, mock_post, status_client):
mock_post.side_effect = RuntimeError("no daemon")
resp = status_client.post("/api/status/refresh", json={})
assert resp.status_code == 500
class TestStatusCancelAll:
@_st("post")
def test_success(self, mock_post, status_client):
from daemon.iface import POST_STATUS_CANCEL_ALL
mock_post.return_value = {"cancelled": ["dnsmasq"], "skipped": {}, "errors": {}}
resp = status_client.post("/api/status/cancel-all")
assert resp.status_code == 200
data = resp.get_json()
assert data["ok"] is True
assert data["data"] == {"cancelled": ["dnsmasq"], "skipped": {}, "errors": {}}
mock_post.assert_called_once_with(POST_STATUS_CANCEL_ALL)
@_st("post")
def test_runtime_error(self, mock_post, status_client):
mock_post.side_effect = RuntimeError("no daemon")
resp = status_client.post("/api/status/cancel-all")
assert resp.status_code == 500
assert resp.get_json()["ok"] is False
+1468
View File
File diff suppressed because it is too large Load Diff
+62
View File
@@ -0,0 +1,62 @@
"""Tests for the daemon-startup filesystem bootstrap (lib.bootstrap)."""
import pytest
from lib import bootstrap, dnsmasq, firewall, network, nginx, wireguard
@pytest.fixture()
def sandbox(tmp_path, monkeypatch):
"""Point every bootstrap-referenced path into a throwaway tree."""
cfg = tmp_path / "config"
data = tmp_path / "data"
monkeypatch.setattr(dnsmasq, "CONFIG_DIR", cfg / "dnsmasq")
monkeypatch.setattr(dnsmasq, "DATA_DIR", data / "dnsmasq")
monkeypatch.setattr(dnsmasq, "FRAGMENTS_DIR", data / "dnsmasq" / "fragments")
monkeypatch.setattr(firewall, "CONFIG_DIR", cfg / "firewall")
monkeypatch.setattr(firewall, "DATA_DIR", data / "firewall")
monkeypatch.setattr(network, "CONFIG_DIR", cfg / "network")
monkeypatch.setattr(network, "DATA_DIR", data / "networkd")
monkeypatch.setattr(nginx, "CONFIG_DIR", cfg / "nginx")
monkeypatch.setattr(nginx, "DATA_DIR", data / "nginx")
monkeypatch.setattr(nginx, "SITES_DIR", data / "nginx" / "sites-enabled")
monkeypatch.setattr(nginx, "CONFIG_FILE", cfg / "nginx" / "config.json")
monkeypatch.setattr(wireguard, "CONFIG_PATH", cfg / "wireguard" / "config.json")
return tmp_path
def test_creates_runtime_dirs(sandbox):
bootstrap.bootstrap()
assert dnsmasq.FRAGMENTS_DIR.is_dir()
assert firewall.DATA_DIR.is_dir()
assert network.DATA_DIR.is_dir()
assert nginx.SITES_DIR.is_dir()
assert wireguard.CONFIG_PATH.parent.is_dir()
def test_does_not_create_config_files(sandbox):
# Config files are left for system-import (first start) or the first
# save_config — bootstrap must not pre-empt either.
bootstrap.bootstrap()
assert not nginx.CONFIG_FILE.exists()
assert not (dnsmasq.CONFIG_DIR / "config.json").exists()
assert not (network.CONFIG_DIR / "config.json").exists()
assert not (firewall.CONFIG_DIR / "config.json").exists()
assert not wireguard.CONFIG_PATH.exists()
def test_persists_nginx_migration(sandbox):
nginx.save_config({"domains": {"app.example.com": {"backend": "myapp"}}})
bootstrap.bootstrap()
on_disk = nginx.get_config()
assert on_disk["backends"]["webui"]["_migrated"] is True
raw = nginx.CONFIG_FILE.read_text()
assert '"_migrated": true' in raw or '"_migrated":True' in raw
def test_idempotent(sandbox):
nginx.save_config({"domains": {}})
bootstrap.bootstrap()
mtime = nginx.CONFIG_FILE.stat().st_mtime_ns
bootstrap.bootstrap()
assert nginx.CONFIG_FILE.stat().st_mtime_ns == mtime
@@ -0,0 +1,100 @@
"""Tests for broadcast_versions per-subsystem contract (daemon.server).
broadcast_versions(subsystem) sends exactly one data-carrying message for
its subsystem no legacy `updated` field and skips the broadcast
entirely when the subsystem's state is None (collector failed).
"""
import asyncio
import json
from unittest.mock import AsyncMock, MagicMock, patch
import daemon.server as server
class TestBroadcastVersionsPerSubsystem:
def _ws(self):
ws = AsyncMock()
ws.send_str = AsyncMock()
server._ws_subscribers.add(ws)
return ws
def test_only_target_subsystem_sent(self):
"""Each subscriber gets one versions message carrying that subsystem."""
store = MagicMock()
store.get.side_effect = lambda name: {"v": name}
ws = self._ws()
try:
with patch.object(server, "state_store", store):
asyncio.run(server.broadcast_versions("firewall"))
ws.send_str.assert_awaited_once()
msg = json.loads(ws.send_str.call_args[0][0])
assert msg == {
"type": "versions",
"subsystem": "firewall",
"data": {"v": "firewall"},
}
finally:
server._ws_subscribers.discard(ws)
def test_data_per_subsystem_not_shared(self):
"""The data payload is that subsystem's state, not another's."""
store = MagicMock()
store.get.side_effect = lambda name: {"name": name}
ws = self._ws()
try:
with patch.object(server, "state_store", store):
asyncio.run(server.broadcast_versions("dnsmasq"))
asyncio.run(server.broadcast_versions("acme"))
msgs = [json.loads(c[0][0]) for c in ws.send_str.call_args_list]
assert [(m["subsystem"], m["data"]) for m in msgs] == [
("dnsmasq", {"name": "dnsmasq"}),
("acme", {"name": "acme"}),
]
# No legacy diff field in any message.
for m in msgs:
assert "updated" not in m
finally:
server._ws_subscribers.discard(ws)
def test_none_state_produces_no_message(self):
"""A None payload (failed collection) is skipped — no clobber."""
store = MagicMock()
store.get.return_value = None
ws = self._ws()
try:
with patch.object(server, "state_store", store):
asyncio.run(server.broadcast_versions("firewall"))
ws.send_str.assert_not_awaited()
finally:
server._ws_subscribers.discard(ws)
def test_no_bump_called(self):
"""broadcast_versions never bumps — callers own the version counter."""
store = MagicMock()
store.get.return_value = {"a": 1}
store.bump = MagicMock()
ws = self._ws()
try:
with patch.object(server, "state_store", store):
asyncio.run(server.broadcast_versions("firewall"))
store.bump.assert_not_called()
finally:
server._ws_subscribers.discard(ws)
def test_dead_subscriber_removed(self):
"""A failing subscriber is pruned and healthy ones still receive data."""
store = MagicMock()
store.get.return_value = {"a": 1}
dead = AsyncMock()
dead.send_str = AsyncMock(side_effect=Exception("broken"))
healthy = self._ws()
server._ws_subscribers.add(dead)
try:
with patch.object(server, "state_store", store):
asyncio.run(server.broadcast_versions("firewall"))
assert dead not in server._ws_subscribers
healthy.send_str.assert_awaited_once()
finally:
server._ws_subscribers.discard(dead)
server._ws_subscribers.discard(healthy)
+201
View File
@@ -0,0 +1,201 @@
"""Tests for lib.common apply-metadata and diff helpers."""
from __future__ import annotations
from lib.common import (
_APPLY_HASH_KEY,
_LAST_APPLIED_CONFIG_KEY,
compute_pending,
config_hash,
deep_diff,
load_json,
revert_to_applied,
save_json,
stamp_applied,
strip_apply_meta,
)
class TestStripApplyMeta:
def test_strips_both_keys(self):
cfg = {"a": 1, _APPLY_HASH_KEY: "h", _LAST_APPLIED_CONFIG_KEY: {}}
assert strip_apply_meta(cfg) == {"a": 1}
def test_missing_keys(self):
assert strip_apply_meta({"a": 1}) == {"a": 1}
def test_does_not_mutate_input(self):
cfg = {"a": 1, _APPLY_HASH_KEY: "h"}
strip_apply_meta(cfg)
assert _APPLY_HASH_KEY in cfg
class TestConfigHashIgnoresMeta:
def test_hash_unaffected_by_metadata(self):
cfg = {"a": 1}
stamped = {"a": 1, _APPLY_HASH_KEY: "x", _LAST_APPLIED_CONFIG_KEY: {"a": 1}}
assert config_hash(cfg) == config_hash(stamped)
class TestStampApplied:
def test_records_snapshot_and_hash(self):
cfg = {"a": 1}
stamp_applied(cfg)
assert cfg[_LAST_APPLIED_CONFIG_KEY] == {"a": 1}
assert cfg[_APPLY_HASH_KEY] == config_hash(cfg)
def test_stable(self):
cfg = {"a": 1}
stamp_applied(cfg)
# A pending-style check: hash matches the current (stripped) config.
assert _APPLY_HASH_KEY in cfg and cfg[_APPLY_HASH_KEY] == config_hash(cfg)
# No drift → no diff.
assert (
deep_diff(cfg.get(_LAST_APPLIED_CONFIG_KEY, {}), strip_apply_meta(cfg))
== []
)
class TestDeepDiff:
def test_identical_empty(self):
assert deep_diff({"a": 1, _APPLY_HASH_KEY: "h"}, {"a": 1}) == []
def test_changed_scalar(self):
diff = deep_diff({"a": 1}, {"a": 2})
assert diff == [{"path": "a", "action": "changed", "old": 1, "new": 2}]
def test_added_removed(self):
added = deep_diff({}, {"a": 1})
assert added[0]["action"] == "added" and added[0]["new"] == 1
removed = deep_diff({"a": 1}, {})
assert removed[0]["action"] == "removed" and removed[0]["old"] == 1
def test_nested_and_list_index(self):
old = {"z": {"svc": ["http"], "ranges": [{"ip": "10.0.0.1", "n": 1}]}}
new = {"z": {"svc": ["http", "ssh"], "ranges": [{"ip": "10.0.0.2", "n": 1}]}}
paths = {d["path"] for d in deep_diff(old, new)}
assert "z.svc" in paths
assert "z.ranges[0].ip" in paths
assert not any(p.startswith("z.ranges[0].n") for p in paths)
class TestComputePending:
def test_hash_match_no_pending(self):
cfg = {"a": 1}
stamp_applied(cfg)
pending, diff = compute_pending(cfg)
assert pending is False
assert diff == []
def test_never_applied_pending_no_snapshot(self):
pending, diff = compute_pending({"a": 1})
assert pending is True
assert diff == []
def test_never_applied_pending_with_foreign_snapshot(self):
# A recorded snapshot that does not match the current hash is still
# used for the diff.
cfg = {"a": 2, _LAST_APPLIED_CONFIG_KEY: {"a": 1}}
pending, diff = compute_pending(cfg)
assert pending is True
assert diff == [{"path": "a", "action": "changed", "old": 1, "new": 2}]
def test_hash_mismatch_with_snapshot_diffs(self):
applied = {"zones": {"lan": {"services": ["http"]}}}
stamped = dict(applied)
stamp_applied(stamped)
drifted = {"zones": {"lan": {"services": ["http", "ssh"]}}}
drifted[_LAST_APPLIED_CONFIG_KEY] = applied
drifted[_APPLY_HASH_KEY] = stamped[_APPLY_HASH_KEY]
pending, diff = compute_pending(drifted)
assert pending is True
paths = {d["path"] for d in diff}
assert "zones.lan.services" in paths
def test_hash_mismatch_snapshot_not_dict(self):
cfg = {"a": 1, _LAST_APPLIED_CONFIG_KEY: "not-a-dict"}
pending, diff = compute_pending(cfg)
assert pending is True
assert diff == []
def test_meta_keys_excluded_from_diff(self):
cfg = {"a": 1}
stamp_applied(cfg)
cfg["a"] = 2 # drift
pending, diff = compute_pending(cfg)
assert pending is True
assert not any(
p.startswith(("_last_applied",)) for d in diff for p in [d["path"]]
)
class TestDashboardFallback:
def test_hash_subsystem_unchanged_generic(self):
# Guards that a pending status without a snapshot still yields a
# renderable pending flag (frontend falls back to a generic line).
status = {"pending_changes": True, "pending_diff": []}
assert status["pending_changes"] is True
assert status["pending_diff"] == []
class TestRevertToApplied:
def test_restores_snapshot_and_clears_pending(self, tmp_path):
path = tmp_path / "config.json"
applied = {"zones": {"lan": {"services": ["http"]}}}
stamped = dict(applied)
stamp_applied(stamped)
# Drift the file after apply (the "pending" state).
dirty = {"zones": {"lan": {"services": ["http", "ssh"]}}}
dirty[_LAST_APPLIED_CONFIG_KEY] = dict(applied)
dirty[_APPLY_HASH_KEY] = stamped[_APPLY_HASH_KEY]
save_json(path, dirty, indent=2)
# Sanity: pending check would report drift.
assert dirty[_APPLY_HASH_KEY] != config_hash(dirty)
ok, reason = revert_to_applied(path)
assert ok and reason == ""
restored = load_json(path)
assert _APPLY_HASH_KEY in restored and restored[_APPLY_HASH_KEY] == config_hash(
restored
)
assert (
_LAST_APPLIED_CONFIG_KEY in restored
and restored[_LAST_APPLIED_CONFIG_KEY] == applied
)
assert (
deep_diff(
restored.get(_LAST_APPLIED_CONFIG_KEY, {}), strip_apply_meta(restored)
)
== []
)
def test_no_baseline(self, tmp_path):
path = tmp_path / "config.json"
save_json(path, {"zones": {}}, indent=2)
ok, reason = revert_to_applied(path)
assert not ok
assert reason
# File untouched.
assert _LAST_APPLIED_CONFIG_KEY not in load_json(path)
def test_missing_file(self, tmp_path):
ok, reason = revert_to_applied(tmp_path / "nope.json")
assert not ok
assert reason
def test_stale_hash_but_snapshot_present(self, tmp_path):
# Baseline recorded, hash stale (drifted) → still revertable.
path = tmp_path / "config.json"
applied = {"a": 1}
stamped = dict(applied)
stamp_applied(stamped)
stamp = dict(stamped)
stamp["a"] = 99 # edited without re-stamping
save_json(path, stamp, indent=2)
assert stamp[_APPLY_HASH_KEY] != config_hash(stamp)
ok, _ = revert_to_applied(path)
assert ok
restored = load_json(path)
assert restored[_APPLY_HASH_KEY] == config_hash(restored)
-81
View File
@@ -70,87 +70,6 @@ class TestSaveConfig:
assert loaded["dns"]["domain"] == "test.lan"
class TestSetDhcpRange:
def test_add_new_range(self, temp_data_dir):
dnsmasq.set_dhcp_range("eth1", "192.168.1.100", "192.168.1.200")
cfg = dnsmasq.get_config()
assert len(cfg["dhcp"]["ranges"]) == 1
assert cfg["dhcp"]["ranges"][0]["interface"] == "eth1"
assert cfg["dhcp"]["ranges"][0]["start"] == "192.168.1.100"
def test_replace_existing_range(self, temp_data_dir):
dnsmasq.set_dhcp_range("eth1", "10.0.0.100", "10.0.0.200")
dnsmasq.set_dhcp_range("eth1", "10.0.0.150", "10.0.0.250")
cfg = dnsmasq.get_config()
assert len(cfg["dhcp"]["ranges"]) == 1
assert cfg["dhcp"]["ranges"][0]["start"] == "10.0.0.150"
class TestStaticLeases:
def test_add_static_lease(self, temp_data_dir):
dnsmasq.add_static_lease("AA:BB:CC:DD:EE:FF", "10.0.0.50", "printer")
cfg = dnsmasq.get_config()
assert len(cfg["dhcp"]["static_leases"]) == 1
assert cfg["dhcp"]["static_leases"][0]["mac"] == "AA:BB:CC:DD:EE:FF"
assert cfg["dhcp"]["static_leases"][0]["hostname"] == "printer"
def test_update_static_lease(self, temp_data_dir):
dnsmasq.add_static_lease("AA:BB:CC", "10.0.0.50")
dnsmasq.add_static_lease("AA:BB:CC", "10.0.0.51")
cfg = dnsmasq.get_config()
assert len(cfg["dhcp"]["static_leases"]) == 1
assert cfg["dhcp"]["static_leases"][0]["ip"] == "10.0.0.51"
def test_remove_static_lease(self, temp_data_dir):
dnsmasq.add_static_lease("AA:BB:CC", "10.0.0.50")
dnsmasq.add_static_lease("11:22:33", "10.0.0.51")
dnsmasq.remove_static_lease("aa:bb:cc")
cfg = dnsmasq.get_config()
assert len(cfg["dhcp"]["static_leases"]) == 1
assert cfg["dhcp"]["static_leases"][0]["mac"] == "11:22:33"
class TestDnsRecords:
def test_add_dns_record(self, temp_data_dir):
dnsmasq.add_dns_record("host", "10.0.0.100")
cfg = dnsmasq.get_config()
assert len(cfg["dns"]["custom_records"]) == 1
def test_remove_dns_record(self, temp_data_dir):
dnsmasq.add_dns_record("host", "10.0.0.100")
dnsmasq.add_dns_record("other", "10.0.0.101")
dnsmasq.remove_dns_record("host")
cfg = dnsmasq.get_config()
assert len(cfg["dns"]["custom_records"]) == 1
assert cfg["dns"]["custom_records"][0]["name"] == "other"
class TestParseLeaseLine:
def test_valid_line(self):
line = "1700000000 AA:BB:CC:DD:EE:FF 10.0.0.50 printer eth1"
result = dnsmasq._parse_lease_line(line)
assert result is not None
assert result["mac"] == "AA:BB:CC:DD:EE:FF"
assert result["ip"] == "10.0.0.50"
assert result["hostname"] == "printer"
def test_empty_line(self):
assert dnsmasq._parse_lease_line("") is None
def test_comment_line(self):
assert dnsmasq._parse_lease_line("# comment") is None
def test_short_line(self):
assert dnsmasq._parse_lease_line("incomplete") is None
def test_minimal_fields(self):
line = "1700000000 AA:BB:CC 10.0.0.50"
result = dnsmasq._parse_lease_line(line)
assert result is not None
assert result["hostname"] == ""
assert result["interface"] == ""
class TestUpstreamsAndDomain:
def test_set_upstreams(self, temp_data_dir):
dnsmasq.set_upstreams(["1.1.1.1", "9.9.9.9"])
+1260 -6
View File
File diff suppressed because it is too large Load Diff
+236 -26
View File
@@ -1,13 +1,16 @@
"""Tests for daemon/handlers/acme.py — handler endpoint logic."""
import asyncio
import inspect
import urllib.error
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
import daemon.handlers.acme as acme_mod
from daemon.handlers.acme import (
IssueRequest,
_check_account_registered,
_check_acme_account,
_check_acme_home_writable,
@@ -25,60 +28,75 @@ from daemon.handlers.acme import (
deactivate_account,
generate_self_signed,
get_account,
get_renew_status,
issue_cert,
register_account,
renew_cert,
)
from daemon.server import ConflictError
from daemon.server import ConflictError, NotFoundError
def _await_renew(body):
"""Start a renewal via renew_cert and await its background task."""
async def _run():
result = renew_cert(None, body)
task = acme_mod._ISSUANCE_TASKS.get(result.get("request_id"))
if task is not None:
await task
return result
return asyncio.run(_run())
class TestGenerateSelfSigned:
def test_generate_creates_files(self, tmp_path):
with (
patch("daemon.handlers.acme._ACME_HOME", tmp_path / "acme"),
patch("daemon.handlers.acme.PROJECT_DIR", tmp_path),
):
result = generate_self_signed(None, {"domain": "test.local"})
assert result["domain"] == "test.local"
assert result["generated"] is True
cert_dir = tmp_path / "acme" / "test.local"
assert result["cert"] == str(cert_dir / "fullchain.cer")
assert result["key"] == str(cert_dir / "test.local.key")
assert (cert_dir / "fullchain.cer").is_file()
assert (cert_dir / "test.local.key").is_file()
certs_dir = tmp_path / "data" / "certs"
assert result["cert"] == str(certs_dir / "test.local.crt")
assert result["key"] == str(certs_dir / "test.local.key")
assert (certs_dir / "test.local.crt").is_file()
assert (certs_dir / "test.local.key").is_file()
def test_generate_idempotent_skips_existing(self, tmp_path):
cert_dir = tmp_path / "acme" / "test.local"
cert_dir.mkdir(parents=True)
(cert_dir / "fullchain.cer").write_text("dummy-cert")
(cert_dir / "test.local.key").write_text("dummy-key")
certs_dir = tmp_path / "data" / "certs"
certs_dir.mkdir(parents=True)
(certs_dir / "test.local.crt").write_text("dummy-cert")
(certs_dir / "test.local.key").write_text("dummy-key")
with patch("daemon.handlers.acme._ACME_HOME", tmp_path / "acme"):
with patch("daemon.handlers.acme.PROJECT_DIR", tmp_path):
result = generate_self_signed(None, {"domain": "test.local"})
assert result["generated"] is False
def test_generate_partial_existing(self, tmp_path):
cert_dir = tmp_path / "acme" / "test.local"
cert_dir.mkdir(parents=True)
(cert_dir / "fullchain.cer").write_text("dummy-cert")
certs_dir = tmp_path / "data" / "certs"
certs_dir.mkdir(parents=True)
(certs_dir / "test.local.crt").write_text("dummy-cert")
# key missing -> should regenerate
with patch("daemon.handlers.acme._ACME_HOME", tmp_path / "acme"):
with patch("daemon.handlers.acme.PROJECT_DIR", tmp_path):
result = generate_self_signed(None, {"domain": "test.local"})
assert result["generated"] is True
def test_generate_custom_days(self, tmp_path):
with (
patch("daemon.handlers.acme._ACME_HOME", tmp_path / "acme"),
patch("daemon.handlers.acme.PROJECT_DIR", tmp_path),
patch("subprocess.run") as mock_run,
):
def _create_files(*args, **kwargs):
cert_dir = tmp_path / "acme" / "test.local"
cert_dir.mkdir(parents=True, exist_ok=True)
(cert_dir / "fullchain.cer").touch()
(cert_dir / "test.local.key").touch()
certs_dir = tmp_path / "data" / "certs"
certs_dir.mkdir(parents=True, exist_ok=True)
(certs_dir / "test.local.crt").touch()
(certs_dir / "test.local.key").touch()
return Path("")
mock_run.side_effect = _create_files
@@ -88,15 +106,15 @@ class TestGenerateSelfSigned:
idx = args.index("-days")
assert args[idx + 1] == "730"
cert_dir = tmp_path / "acme" / "test.local"
if (cert_dir / "fullchain.cer").is_file():
assert cert_dir.is_dir()
certs_dir = tmp_path / "data" / "certs"
if (certs_dir / "test.local.crt").is_file():
assert certs_dir.is_dir()
def test_generate_creates_directory(self, tmp_path):
with patch("daemon.handlers.acme._ACME_HOME", tmp_path / "acme"):
with patch("daemon.handlers.acme.PROJECT_DIR", tmp_path):
generate_self_signed(None, {"domain": "test.local"})
assert (tmp_path / "acme" / "test.local").is_dir()
assert (tmp_path / "data" / "certs").is_dir()
def test_generate_requires_domain(self):
with pytest.raises(ValueError, match="domain"):
@@ -1165,3 +1183,195 @@ class TestIssueCertExistingCerts:
assert result["domain"] == "example.com"
assert "request_id" in result
mock_run_issue.assert_called_once()
class TestRenewCert:
def test_requires_body(self):
with pytest.raises(ValueError, match="Request body required"):
renew_cert(None, None)
def test_requires_domain(self):
with pytest.raises(ValueError, match="'domain' is required"):
renew_cert(None, {"force": True})
def test_starts_background_task(self):
with (
patch.dict("daemon.handlers.acme._ISSUANCES", clear=True),
patch.dict("daemon.handlers.acme._ISSUANCE_TASKS", clear=True),
patch(
"daemon.handlers.acme._run_acme", return_value="Renewed 'example.com'"
),
patch("daemon.handlers.acme.refresh_state") as mock_refresh,
):
result = _await_renew({"domain": "example.com"})
req = acme_mod._ISSUANCES[result["request_id"]]
assert result["domain"] == "example.com"
assert "request_id" in result
assert req.status == "completed"
assert [s.status for s in req.steps] == ["done", "done", "done"]
mock_refresh.assert_called_once_with(["acme"])
def test_skip_when_not_due(self):
output = (
"[Thu Aug 20 01:27:41 AM UTC 2026] Skipping. "
"Next renewal time is: 1785068261 (2026-07-26T12:17:41Z)"
)
with (
patch.dict("daemon.handlers.acme._ISSUANCES", clear=True),
patch.dict("daemon.handlers.acme._ISSUANCE_TASKS", clear=True),
patch("daemon.handlers.acme._run_acme", return_value=output),
patch("daemon.handlers.acme.refresh_state") as mock_refresh,
):
result = _await_renew({"domain": "example.com"})
req = acme_mod._ISSUANCES[result["request_id"]]
assert req.status == "skipped"
assert req.steps[0].status == "done"
mock_refresh.assert_not_called()
def test_failure_marks_failed(self):
with (
patch.dict("daemon.handlers.acme._ISSUANCES", clear=True),
patch.dict("daemon.handlers.acme._ISSUANCE_TASKS", clear=True),
patch(
"daemon.handlers.acme._run_acme",
side_effect=RuntimeError("acme.sh failed with exit code 1: boom"),
),
):
result = _await_renew({"domain": "example.com"})
req = acme_mod._ISSUANCES[result["request_id"]]
assert req.status == "failed"
assert req.steps[0].status == "error"
assert "boom" in req.steps[0].message
def test_force_appends_flag(self):
with (
patch.dict("daemon.handlers.acme._ISSUANCES", clear=True),
patch.dict("daemon.handlers.acme._ISSUANCE_TASKS", clear=True),
patch("daemon.handlers.acme._run_acme", return_value="ok") as mock_acme,
patch("daemon.handlers.acme.refresh_state"),
):
_await_renew({"domain": "example.com", "force": True})
renew_args = mock_acme.call_args_list[0].args[0]
assert "--force" in renew_args
def test_dedup_running_returns_existing(self):
existing = IssueRequest(request_id="existing123", domain="example.com")
with (
patch.dict("daemon.handlers.acme._ISSUANCES", clear=True),
patch.dict("daemon.handlers.acme._ISSUANCE_TASKS", clear=True),
patch("daemon.handlers.acme._run_acme") as mock_acme,
):
acme_mod._ISSUANCES["existing123"] = existing
result = renew_cert(None, {"domain": "example.com"})
assert result == {
"request_id": "existing123",
"domain": "example.com",
"status": "existing",
}
mock_acme.assert_not_called()
class TestGetRenewStatus:
def test_missing_id(self):
with pytest.raises(ValueError, match="'id' is required"):
get_renew_status(None, None)
with pytest.raises(ValueError, match="'id' is required"):
get_renew_status(None, {})
def test_unknown_id(self):
with (
patch.dict("daemon.handlers.acme._ISSUANCES", clear=True),
pytest.raises(NotFoundError, match="not found"),
):
get_renew_status(None, {"id": "unknown"})
def test_returns_request_dict(self):
with (
patch.dict("daemon.handlers.acme._ISSUANCES", clear=True),
patch.dict("daemon.handlers.acme._ISSUANCE_TASKS", clear=True),
patch("daemon.handlers.acme._run_acme", return_value="Renewed"),
patch("daemon.handlers.acme.refresh_state"),
):
result = _await_renew({"domain": "example.com"})
status = get_renew_status(None, {"id": result["request_id"]})
assert status["request_id"] == result["request_id"]
assert status["domain"] == "example.com"
assert status["status"] == "completed"
assert [s["name"] for s in status["steps"]] == ["renew", "deploy", "refresh"]
class TestNormalizeAcmeHome:
def test_normalize_invokes_sudo_chmod_on_files(self, tmp_path):
f1 = tmp_path / "account.conf"
f1.write_text("x")
(tmp_path / "sub").mkdir()
f2 = tmp_path / "sub" / "dom.key"
f2.write_text("x")
with (
patch("daemon.handlers.acme._ACME_HOME", tmp_path),
patch(
"lib.common.run_proc",
return_value=MagicMock(returncode=0, stderr=""),
) as mock_proc,
):
acme_mod.normalize_acme_home()
args = mock_proc.call_args.args[0]
assert args[:2] == ["chmod", "g+rwX"]
assert set(args[2:]) == {str(f1), str(f2)}
mock_proc.assert_called_once()
def test_normalize_empty_tree_skips_sudo(self, tmp_path):
with (
patch("daemon.handlers.acme._ACME_HOME", tmp_path),
patch("lib.common.run_proc") as mock_proc,
):
acme_mod.normalize_acme_home()
mock_proc.assert_not_called()
def test_normalize_failure_does_not_raise(self, tmp_path):
(tmp_path / "a.conf").write_text("x")
with (
patch("daemon.handlers.acme._ACME_HOME", tmp_path),
patch(
"lib.common.run_proc",
return_value=MagicMock(returncode=1, stderr="denied"),
),
patch.object(acme_mod, "logger"),
):
acme_mod.normalize_acme_home()
def test_preflight_normalizes_before_run(self):
calls = []
with (
patch.object(
acme_mod,
"normalize_acme_home",
side_effect=lambda: calls.append("normalize"),
),
patch.object(
acme_mod,
"_run_acme",
side_effect=lambda args: calls.append("run:" + " ".join(args)) or "ok",
),
):
out = acme_mod._run_acme_preflight(["--list", "--listraw"])
assert calls == ["normalize", "run:--list --listraw"]
assert out == "ok"
class TestPreflightWiring:
def test_issue_uses_preflight(self):
source = inspect.getsource(acme_mod._run_issue)
assert "_run_acme_preflight" in source
assert "normalize_acme_home" in source
def test_renew_uses_preflight(self):
source = inspect.getsource(acme_mod._run_renew)
assert "_run_acme_preflight" in source
assert "normalize_acme_home" in source
+39 -5
View File
@@ -15,18 +15,36 @@ from daemon.handlers.network import (
save_interface,
set_sysctl,
)
from lib import dnsmasq as _dm
from lib import firewall as _fw
from lib import network as _net
@pytest.fixture
def tmp_network(tmp_path):
orig_config = _net.CONFIG_FILE
orig_data = _net.DATA_DIR
# Handler endpoints emit "networkd" sync events; the subscribers
# (lib.sync.NetworkToAllSync) read/write the firewall and dnsmasq
# configs, and apply_all re-stamps the dnsmasq config. Point all of
# those paths at tmp so tests never touch the real config files.
orig_net = (_net.CONFIG_FILE, _net.DATA_DIR)
orig_dm = (_dm.CONFIG_DIR, _dm.DATA_DIR, _dm.CONFIG_PATH, _dm.FRAGMENTS_DIR)
orig_fw = (_fw.CONFIG_DIR, _fw.CONFIG_FILE)
_net.CONFIG_FILE = tmp_path / "config" / "network" / "config.json"
_net.DATA_DIR = tmp_path / "data" / "networkd"
_dm.CONFIG_DIR = tmp_path / "config" / "dnsmasq"
_dm.DATA_DIR = tmp_path / "data" / "dnsmasq"
_dm.CONFIG_PATH = _dm.CONFIG_DIR / "config.json"
_dm.FRAGMENTS_DIR = _dm.DATA_DIR / "fragments"
_dm.CONFIG_DIR.mkdir(parents=True, exist_ok=True)
_dm.DATA_DIR.mkdir(parents=True, exist_ok=True)
_dm.FRAGMENTS_DIR.mkdir(parents=True, exist_ok=True)
_fw.CONFIG_DIR = tmp_path / "config" / "firewall"
_fw.CONFIG_FILE = _fw.CONFIG_DIR / "config.json"
_fw.CONFIG_DIR.mkdir(parents=True, exist_ok=True)
yield tmp_path
_net.CONFIG_FILE = orig_config
_net.DATA_DIR = orig_data
_net.CONFIG_FILE, _net.DATA_DIR = orig_net
_dm.CONFIG_DIR, _dm.DATA_DIR, _dm.CONFIG_PATH, _dm.FRAGMENTS_DIR = orig_dm
_fw.CONFIG_DIR, _fw.CONFIG_FILE = orig_fw
# =================================================================
@@ -160,10 +178,14 @@ class TestApplyAll:
}
)
runtime_dir = tmp_network / "run" / "vacuum-wall"
runtime_dir.mkdir(parents=True)
with (
patch("daemon.handlers.network.generate_network_files") as mock_gen,
patch("daemon.handlers.network.run") as mock_run,
patch("daemon.handlers.network.collect_upstream_dns", return_value=[]),
patch("daemon.handlers.network.RUNTIME_DIR", runtime_dir),
):
mock_gen.return_value = {
"generated": [_net.DATA_DIR / "99-eth0.network"],
@@ -188,11 +210,15 @@ class TestApplyAll:
}
)
runtime_dir = tmp_network / "run" / "vacuum-wall"
runtime_dir.mkdir(parents=True)
with (
patch("daemon.handlers.network.generate_network_files") as mock_gen,
patch("daemon.handlers.network.run") as mock_run,
patch("daemon.handlers.network.set_upstreams") as mock_set_upstreams,
patch("daemon.handlers.network.collect_upstream_dns") as mock_collect,
patch("daemon.handlers.network.RUNTIME_DIR", runtime_dir),
):
mock_gen.return_value = {
"generated": [_net.DATA_DIR / "99-eth0.network"],
@@ -208,6 +234,9 @@ class TestApplyAll:
def test_apply_all_handles_dns_sync_failure(self, tmp_network):
_net.save_config({"interfaces": {"eth0": {"dns": ["8.8.8.8"]}}})
runtime_dir = tmp_network / "run" / "vacuum-wall"
runtime_dir.mkdir(parents=True)
with (
patch("daemon.handlers.network.generate_network_files") as mock_gen,
patch("daemon.handlers.network.run") as mock_run,
@@ -216,6 +245,7 @@ class TestApplyAll:
side_effect=RuntimeError("fail"),
),
patch("daemon.handlers.network.collect_upstream_dns") as mock_collect,
patch("daemon.handlers.network.RUNTIME_DIR", runtime_dir),
):
mock_gen.return_value = {
"generated": [_net.DATA_DIR / "99-eth0.network"],
@@ -233,10 +263,14 @@ class TestApplyAll:
sys_dir.mkdir(parents=True)
(sys_dir / "stale-file.network").write_text("[Match]\nName=old\n")
runtime_dir = tmp_network / "run" / "vacuum-wall"
runtime_dir.mkdir(parents=True)
with (
patch("daemon.handlers.network.generate_network_files") as mock_gen,
patch("daemon.handlers.network.run") as mock_run,
patch("daemon.handlers.network.collect_upstream_dns", return_value=[]),
patch("daemon.handlers.network.RUNTIME_DIR", runtime_dir),
):
mock_gen.return_value = {
"generated": [_net.DATA_DIR / "99-eth0.network"],
@@ -347,7 +381,7 @@ class TestInferEndpoints:
class TestSetSysctl:
def test_set_sysctl_success(self):
def test_set_sysctl_success(self, tmp_network):
with (
patch("daemon.handlers.network.run") as mock_run,
patch.object(Path, "read_text", return_value="1"),
+3 -2
View File
@@ -29,10 +29,11 @@ class TestGetConfig:
assert isinstance(cfg, dict)
assert "interfaces" in cfg
def test_creates_config_file(self, tmp_network):
def test_missing_file_returns_default_without_writing(self, tmp_network):
# Pure read: get_config never materializes the file.
cfg = _net.get_config()
assert _net.CONFIG_FILE.exists()
assert cfg["interfaces"] == {}
assert not _net.CONFIG_FILE.exists()
class TestSaveConfig:
+12 -10
View File
@@ -279,18 +279,18 @@ class TestStateParserDedup:
"""Verify lib/state.py uses lib.network.parse_networkctl_status()."""
def test_state_uses_network_parser(self):
"""The networkd collector in state.py should import from lib.network."""
import lib.state as _state
"""The networkd collector should import from lib.network."""
import daemon.collectors.networkd as _collector
source = Path(_state.__file__).read_text()
assert "from lib.network import parse_networkctl_status" in source
source = Path(_collector.__file__).read_text()
assert "from lib.network import" in source
assert "parse_networkctl_status" in source
def test_networkd_collector_returns_correct_format(self):
"""_collect_networkd should return interfaces dict + timestamp."""
import lib.state as _state
import daemon.collectors.networkd as _collector
with patch("lib.state.run") as mock_run:
with patch("daemon.collectors.networkd.run") as mock_run:
mock_run.return_value = json.dumps(
{
"Interfaces": [
@@ -320,7 +320,7 @@ class TestStateParserDedup:
]
}
)
result = _state._collect_networkd()
result = _collector._collect_networkd()
assert "interfaces" in result
assert "timestamp" in result
@@ -329,10 +329,12 @@ class TestStateParserDedup:
def test_networkd_collector_handles_failure(self):
"""_collect_networkd returns empty interfaces on error."""
import lib.state as _state
import daemon.collectors.networkd as _collector
with patch("lib.state.run", side_effect=RuntimeError("no networkctl")):
result = _state._collect_networkd()
with patch(
"daemon.collectors.networkd.run", side_effect=RuntimeError("no networkctl")
):
result = _collector._collect_networkd()
assert result["interfaces"] == {}
assert "timestamp" in result
+733 -26
View File
@@ -55,17 +55,69 @@ class TestGetConfig:
assert "ssl" in cfg
assert cfg["domains"] == {}
def test_read_does_not_rewrite_unchanged_file(self, temp_data_dir):
"""get_config() must not re-save a file that needs no migration."""
nginx.save_config(
{
"backends": {"webui": {"_migrated": True, "paths": {}}},
"domains": {"app.example.com": {"backend": "webui"}},
"ssl": {"protocols": "TLSv1.3"},
}
)
mtime_before = nginx.CONFIG_FILE.stat().st_mtime_ns
cfg = nginx.get_config()
assert cfg["domains"] == {"app.example.com": {"backend": "webui"}}
# No churn: reading a current-format config leaves the file alone.
assert nginx.CONFIG_FILE.stat().st_mtime_ns == mtime_before
def test_read_migrates_in_memory_without_writing(self, temp_data_dir):
"""get_config() is pure: migration is applied in memory, file untouched."""
nginx.save_config({"domains": {"app.example.com": {"backend": "myapp"}}})
mtime_before = nginx.CONFIG_FILE.stat().st_mtime_ns
cfg = nginx.get_config()
# Migration added the builtin webui backend (in memory only).
assert cfg["backends"]["webui"]["_migrated"] is True
assert nginx.CONFIG_FILE.stat().st_mtime_ns == mtime_before
def test_migrate_config_file_persists_legacy(self, temp_data_dir):
"""migrate_config_file() rewrites the file when migration changes it."""
nginx.save_config({"domains": {"app.example.com": {"backend": "myapp"}}})
mtime_before = nginx.CONFIG_FILE.stat().st_mtime_ns
assert nginx.migrate_config_file() is True
assert nginx.CONFIG_FILE.stat().st_mtime_ns != mtime_before
# Idempotent: a second run is a no-op.
assert nginx.migrate_config_file() is False
def test_migrate_config_file_noop_when_missing(self, temp_data_dir):
assert not nginx.CONFIG_FILE.exists()
assert nginx.migrate_config_file() is False
assert not nginx.CONFIG_FILE.exists()
class TestSaveConfig:
def test_saves_and_reloads(self, temp_data_dir):
cfg = {
"domains": {"example.com": {"backend": {"host": "localhost", "port": 80}}}
"backends": {
"myapp": {
"label": "My App",
"paths": {
"/": {
"backend": {
"host": "localhost",
"port": 80,
"proto": "http",
}
}
},
}
},
"domains": {"example.com": {"backend": "myapp"}},
}
nginx.save_config(cfg)
loaded = nginx.get_config()
assert loaded["domains"]["example.com"]["backend"] == "myapp"
assert (
loaded["domains"]["example.com"]["paths"]["/"]["backend"]["host"]
== "localhost"
loaded["backends"]["myapp"]["paths"]["/"]["backend"]["host"] == "localhost"
)
@@ -76,46 +128,76 @@ class TestGetDomains:
def test_returns_domain_list(self, temp_data_dir):
cfg = {
"domains": {
"example.com": {
"backend": {"host": "localhost", "port": 8080, "proto": "http"},
"force_ssl": True,
"backends": {
"myapp": {
"label": "My App",
"paths": {
"/": {
"backend": {
"host": "localhost",
"port": 8080,
"proto": "http",
}
},
"/api": {
"backend": {
"host": "localhost",
"port": 8081,
"proto": "http",
}
},
},
}
}
},
"domains": {
"example.com": {"backend": "myapp", "force_ssl": True},
},
}
nginx.save_config(cfg)
result = nginx.get_domains()
assert len(result) == 1
assert len(result) == 2
assert result[0]["domain"] == "example.com"
assert result[0]["backend_name"] == "myapp"
assert result[0]["path"] == "/"
class TestAddDomain:
@patch("lib.nginx.get_config")
def test_add_domain(self, mock_get, temp_data_dir):
mock_get.return_value = {
def test_add_domain(self, temp_data_dir):
cfg = {
"backends": {
"myapp": {
"label": "My App",
"paths": {
"/": {
"backend": {
"host": "10.0.0.5",
"port": 8080,
"proto": "http",
}
}
},
}
},
"domains": {},
"management": None,
"ssl": {"protocols": "TLSv1.2 TLSv1.3"},
}
nginx.add_domain("example.com", "10.0.0.5", 8080)
cfg = nginx.get_config()
assert "example.com" in cfg["domains"]
assert (
cfg["domains"]["example.com"]["paths"]["/"]["backend"]["host"] == "10.0.0.5"
)
assert cfg["domains"]["example.com"]["paths"]["/"]["backend"]["port"] == 8080
nginx.save_config(cfg)
nginx.add_domain("example.com", "myapp")
loaded = nginx.get_config()
assert "example.com" in loaded["domains"]
assert loaded["domains"]["example.com"]["backend"] == "myapp"
assert loaded["domains"]["example.com"]["force_ssl"] is True
@patch("lib.nginx.get_config")
def test_duplicate_domain_raises(self, mock_get, temp_data_dir):
mock_get.return_value = {
"domains": {
"example.com": {"backend": {"host": "x", "port": 80, "proto": "http"}}
},
"management": None,
"backends": {"myapp": {}},
"domains": {"example.com": {"backend": "myapp"}},
"ssl": {"protocols": "TLSv1.2 TLSv1.3"},
}
nginx.save_config(mock_get.return_value)
with pytest.raises(ValueError):
nginx.add_domain("example.com", "10.0.0.5", 8080)
nginx.add_domain("example.com", "myapp")
class TestRemoveDomain:
@@ -222,9 +304,71 @@ class TestGenerateServerConf:
}
out = nginx.generate_server_conf(cfg)
assert "proxy_pass http://127.0.0.1:9090;" in out
assert "add_header X-Content-Type-Options" not in out
# Server-level security headers come from Flask, not nginx
assert "Strict-Transport-Security" not in out
assert "Referrer-Policy" not in out
assert "wall_mgmt_access.log" in out
def test_management_static_location(self, temp_data_dir):
cfg = {
"domain": "mgmt.example.com",
"paths": {
"/": {
"backend": {"host": "127.0.0.1", "port": 9090, "proto": "http"},
"is_management": True,
},
"/ws": {
"backend": {"host": "127.0.0.1", "port": 9091, "proto": "http"},
"is_websocket": True,
},
},
"force_ssl": True,
"cert": "acme",
}
out = nginx.generate_server_conf(cfg)
static_root = str(nginx.PROJECT_DIR / "webui" / "static")
assert "location /static/ {" in out
assert f"alias {static_root}/;" in out
assert 'add_header Cache-Control "no-cache" always;' in out
assert "add_header X-Content-Type-Options nosniff always;" in out
assert (
"add_header Content-Security-Policy \"default-src 'none'\" always;" in out
)
def test_static_location_only_for_management_root(self, temp_data_dir):
cfg = {
"domain": "app.example.com",
"paths": {
"/": {
"backend": {"host": "10.0.0.1", "port": 80, "proto": "http"},
}
},
"force_ssl": True,
"cert": "acme",
}
out = nginx.generate_server_conf(cfg)
assert "location /static/" not in out
def test_management_static_location_on_subpath(self, temp_data_dir):
# The SPA references /static/... at the domain root regardless of the
# management backend path, so the block is emitted for any
# is_management path, not only '/'.
cfg = {
"domain": "mgmt.example.com",
"paths": {
"/app": {
"backend": {"host": "127.0.0.1", "port": 9090, "proto": "http"},
"is_management": True,
},
},
"force_ssl": True,
"cert": "acme",
}
out = nginx.generate_server_conf(cfg)
assert "location /static/ {" in out
static_root = str(nginx.PROJECT_DIR / "webui" / "static")
assert f"alias {static_root}/;" in out
def test_websocket_path(self, temp_data_dir):
cfg = {
"domain": "mgmt.example.com",
@@ -380,3 +524,566 @@ class TestHashPasswordFallback:
result = nginx._hash_password("test")
assert isinstance(result, str)
assert len(result) > 0
# ---------------------------------------------------------------------------
# Backend CRUD and resolution tests (daemon handler)
# NOTE: The daemon handler modules cannot be called as-is because they
# require sudo and live service paths. Instead we test through `lib.nginx`
# public API where possible, and unit-test the handler-internal functions
# by importing them directly.
# ---------------------------------------------------------------------------
class TestResolvePaths:
def test_resolves_from_backend(self, temp_data_dir):
"""Paths resolve from backends[domain_cfg['backend']].paths."""
backends = {
"myapp": {
"label": "My App",
"paths": {
"/": {"backend": {"host": "a", "port": 80, "proto": "http"}},
"/api": {"backend": {"host": "b", "port": 8080, "proto": "http"}},
},
}
}
domain_cfg = {"backend": "myapp"}
resolved = nginx._resolve_paths(domain_cfg, backends)
assert resolved == backends["myapp"]["paths"]
def test_returns_empty_when_backend_missing(self, temp_data_dir):
"""Empty dict when backend ref not found in backends."""
resolved = nginx._resolve_paths({"backend": "nonexistent"}, {})
assert resolved == {}
def test_returns_empty_when_no_backend_key(self, temp_data_dir):
"""Empty dict when domain has no backend key."""
resolved = nginx._resolve_paths({}, {"myapp": {"paths": {"/": {}}}})
assert resolved == {}
class TestResolveAuth:
def test_domain_auth_wins(self, temp_data_dir):
"""Domain-level auth overrides backend auth."""
backends = {
"myapp": {"auth": {"user": "backend", "htpasswd": "/backend/.htpasswd"}}
}
domain_cfg = {
"backend": "myapp",
"auth": {"user": "domain", "htpasswd": "/domain/.htpasswd"},
}
resolved = nginx._resolve_auth(domain_cfg, backends)
assert resolved == {"user": "domain", "htpasswd": "/domain/.htpasswd"}
def test_domain_auth_null_disables(self, temp_data_dir):
"""Domain auth set to None disables all auth."""
backends = {
"myapp": {"auth": {"user": "backend", "htpasswd": "/backend/.htpasswd"}}
}
domain_cfg = {"backend": "myapp", "auth": None}
resolved = nginx._resolve_auth(domain_cfg, backends)
assert resolved is None
def test_backend_auth_applies(self, temp_data_dir):
"""Backend auth applies when domain has no auth key."""
backends = {
"myapp": {"auth": {"user": "backend", "htpasswd": "/backend/.htpasswd"}}
}
domain_cfg = {"backend": "myapp"}
resolved = nginx._resolve_auth(domain_cfg, backends)
assert resolved == {"user": "backend", "htpasswd": "/backend/.htpasswd"}
def test_no_auth_when_absent_everywhere(self, temp_data_dir):
"""None when neither domain nor backend define auth."""
backends = {"myapp": {"paths": {}}}
domain_cfg = {"backend": "myapp"}
resolved = nginx._resolve_auth(domain_cfg, backends)
assert resolved is None
class TestBackendCRUD:
"""Test daemon handler backend CRUD operations directly."""
def test_validate_paths_valid(self, temp_data_dir):
"""_validate_paths accepts correct schemas."""
from daemon.handlers.nginx import _validate_paths
paths = {
"/": {"backend": {"host": "a", "port": 80, "proto": "http"}},
"/ws": {
"backend": {"host": "b", "port": 9091, "proto": "http"},
"is_websocket": True,
},
}
_validate_paths(paths) # no exception
def test_validate_paths_missing_host(self, temp_data_dir):
"""_validate_paths raises when backend.host missing."""
from daemon.handlers.nginx import _validate_paths
with pytest.raises(ValueError, match="missing 'host'"):
_validate_paths({"/": {"backend": {"port": 80, "proto": "http"}}})
def test_validate_paths_missing_port(self, temp_data_dir):
"""_validate_paths raises when backend.port missing."""
from daemon.handlers.nginx import _validate_paths
with pytest.raises(ValueError, match="missing 'port'"):
_validate_paths({"/": {"backend": {"host": "a", "proto": "http"}}})
def test_validate_paths_missing_proto(self, temp_data_dir):
"""_validate_paths raises when backend.proto missing."""
from daemon.handlers.nginx import _validate_paths
with pytest.raises(ValueError, match="missing 'proto'"):
_validate_paths({"/": {"backend": {"host": "a", "port": 80}}})
def test_validate_paths_no_backend(self, temp_data_dir):
"""_validate_paths raises when path lacks backend dict."""
from daemon.handlers.nginx import _validate_paths
with pytest.raises(ValueError, match="missing 'backend'"):
_validate_paths({"/": {"something": "else"}})
@patch("daemon.handlers.nginx._get_config")
@patch("daemon.handlers.nginx._save_config")
def test_add_backend(self, mock_save, mock_get, temp_data_dir):
"""_add_backend creates a new backend entry."""
from daemon.handlers.nginx import _add_backend
mock_get.return_value = {
"backends": {},
"domains": {},
"ssl": {},
}
_add_backend(
"test",
"Test Label",
{
"/": {"backend": {"host": "a", "port": 80, "proto": "http"}},
},
)
saved_cfg = mock_save.call_args[0][0]
assert "test" in saved_cfg["backends"]
assert saved_cfg["backends"]["test"]["label"] == "Test Label"
@patch("daemon.handlers.nginx._get_config")
@patch("daemon.handlers.nginx._save_config")
def test_add_backend_duplicate_raises(self, mock_save, mock_get, temp_data_dir):
"""_add_backend raises ValueError for duplicate name."""
from daemon.handlers.nginx import _add_backend
mock_get.return_value = {
"backends": {"test": {"label": "Existing"}},
"domains": {},
"ssl": {},
}
with pytest.raises(ValueError, match="already exists"):
_add_backend(
"test",
"New Label",
{
"/": {"backend": {"host": "a", "port": 80, "proto": "http"}},
},
)
@patch("daemon.handlers.nginx._get_config")
@patch("daemon.handlers.nginx._save_config")
def test_update_backend(self, mock_save, mock_get, temp_data_dir):
"""_update_backend modifies label and paths."""
from daemon.handlers.nginx import _update_backend
mock_get.return_value = {
"backends": {
"test": {
"label": "Old",
"paths": {
"/": {"backend": {"host": "a", "port": 80, "proto": "http"}}
},
}
},
"domains": {},
"ssl": {},
}
_update_backend(
"test",
label="New",
paths={
"/api": {"backend": {"host": "b", "port": 9000, "proto": "http"}},
},
)
saved_cfg = mock_save.call_args[0][0]
assert saved_cfg["backends"]["test"]["label"] == "New"
assert "/api" in saved_cfg["backends"]["test"]["paths"]
@patch("daemon.handlers.nginx._get_config")
def test_update_backend_notfound_raises(self, mock_get, temp_data_dir):
"""_update_backend raises KeyError for unknown backend."""
from daemon.handlers.nginx import _update_backend
mock_get.return_value = {"backends": {}, "domains": {}, "ssl": {}}
with pytest.raises(KeyError):
_update_backend("nonexistent", label="X")
@patch("daemon.handlers.nginx._get_config")
def test_update_backend_builtin_raises(self, mock_get, temp_data_dir):
"""_update_backend raises ValueError for builtin backend."""
from daemon.handlers.nginx import _update_backend
mock_get.return_value = {
"backends": {"webui": {"label": "WebUI", "builtin": True, "paths": {}}},
"domains": {},
"ssl": {},
}
with pytest.raises(ValueError, match="Cannot modify"):
_update_backend("webui", label="Hacked")
@patch("daemon.handlers.nginx._get_config")
@patch("daemon.handlers.nginx._save_config")
def test_remove_backend(self, mock_save, mock_get, temp_data_dir):
"""_remove_backend deletes non-builtin backend."""
from daemon.handlers.nginx import _remove_backend
mock_get.return_value = {
"backends": {"test": {"label": "Test", "paths": {}}},
"domains": {},
"ssl": {},
}
_remove_backend("test")
saved_cfg = mock_save.call_args[0][0]
assert "test" not in saved_cfg["backends"]
@patch("daemon.handlers.nginx._get_config")
def test_remove_backend_notfound_raises(self, mock_get, temp_data_dir):
"""_remove_backend raises KeyError for unknown backend."""
from daemon.handlers.nginx import _remove_backend
mock_get.return_value = {"backends": {}, "domains": {}, "ssl": {}}
with pytest.raises(KeyError):
_remove_backend("nonexistent")
@patch("daemon.handlers.nginx._get_config")
def test_remove_backend_builtin_raises(self, mock_get, temp_data_dir):
"""_remove_backend raises ValueError for builtin backend."""
from daemon.handlers.nginx import _remove_backend
mock_get.return_value = {
"backends": {"webui": {"label": "WebUI", "builtin": True, "paths": {}}},
"domains": {},
"ssl": {},
}
with pytest.raises(ValueError, match="Cannot remove"):
_remove_backend("webui")
@patch("daemon.handlers.nginx._get_config")
def test_remove_backend_referenced_raises_conflict(self, mock_get, temp_data_dir):
"""_remove_backend raises ConflictError when domains reference it."""
from daemon.handlers.nginx import _remove_backend
from daemon.server import ConflictError
mock_get.return_value = {
"backends": {"myapp": {"label": "App", "paths": {}}},
"domains": {"example.com": {"backend": "myapp"}},
"ssl": {},
}
with pytest.raises(ConflictError, match="referenced by domain"):
_remove_backend("myapp")
class TestMigration:
def test_ensure_webui_backend_creates(self, temp_data_dir):
"""_ensure_webui_backend creates webui backend if missing."""
cfg = {}
nginx._ensure_webui_backend(cfg)
assert "webui" in cfg["backends"]
assert cfg["backends"]["webui"]["_migrated"] is True
assert cfg["backends"]["webui"]["label"] == "Vacuum Wall WebUI"
assert cfg["backends"]["webui"]["builtin"] is True
def test_ensure_webui_backend_skips_migrated(self, temp_data_dir):
"""_ensure_webui_backend skips if _migrated is true."""
cfg = {
"backends": {
"webui": {
"label": "Custom",
"_migrated": True,
"paths": {},
}
}
}
nginx._ensure_webui_backend(cfg)
# Label unchanged — not recreated
assert cfg["backends"]["webui"]["label"] == "Custom"
def test_migrate_mgmt_domains_application_webui(self, temp_data_dir):
"""Domains with application='webui' and mgmt paths get backend='webui'."""
cfg = {
"backends": {"webui": {"_migrated": True, "paths": {}}},
"domains": {
"mgmt.example.com": {
"application": "webui",
"paths": {
"/": {"is_management": True},
"/ws": {"is_websocket": True},
},
"auth": {"user": "admin"},
}
},
}
nginx._migrate_mgmt_domains(cfg)
dom = cfg["domains"]["mgmt.example.com"]
assert dom["backend"] == "webui"
assert "application" not in dom
assert "paths" not in dom
assert "auth" not in dom
def test_migrate_mgmt_domains_strips_application_only(self, temp_data_dir):
"""application='webui' stripped even when paths don't match mgmt shape."""
cfg = {
"backends": {"webui": {"_migrated": True, "paths": {}}},
"domains": {
"odd.example.com": {
"application": "webui",
"paths": {"/": {"is_management": True}},
}
},
}
nginx._migrate_mgmt_domains(cfg)
dom = cfg["domains"]["odd.example.com"]
assert "application" not in dom
assert "backend" not in dom # paths didn't match full mgmt shape
assert "paths" in dom # not removed
def test_migrate_mgmt_domains_detects_webui_paths(self, temp_data_dir):
"""Domains matching webui path shape get migrated to backend."""
cfg = {
"backends": {"webui": {"_migrated": True, "paths": {}}},
"domains": {
"mgmt.example.com": {
"paths": {
"/": {
"backend": {
"host": "127.0.0.1",
"port": 9090,
"proto": "http",
},
"is_management": True,
},
"/ws": {
"backend": {
"host": "127.0.0.1",
"port": 9091,
"proto": "http",
},
"is_websocket": True,
},
}
}
},
}
nginx._migrate_mgmt_domains(cfg)
dom = cfg["domains"]["mgmt.example.com"]
assert dom["backend"] == "webui"
assert "paths" not in dom
def test_migrate_mgmt_domains_skips_non_mgmt(self, temp_data_dir):
"""Non-management domains are left untouched."""
cfg = {
"backends": {"webui": {"_migrated": True, "paths": {}}},
"domains": {
"app.example.com": {
"paths": {
"/": {
"backend": {"host": "10.0.0.1", "port": 80, "proto": "http"}
}
}
}
},
}
nginx._migrate_mgmt_domains(cfg)
dom = cfg["domains"]["app.example.com"]
assert "backend" not in dom
assert "paths" in dom
def test_migrate_config_full(self, temp_data_dir):
"""_migrate_config runs _ensure_webui_backend then _migrate_mgmt_domains."""
cfg = {
"domains": {
"mgmt.example.com": {
"application": "webui",
"paths": {
"/": {"is_management": True},
"/ws": {"is_websocket": True},
},
}
}
}
result = nginx._migrate_config(cfg)
assert result is cfg
assert "webui" in cfg["backends"]
assert cfg["domains"]["mgmt.example.com"]["backend"] == "webui"
class TestDomainSwap:
"""Domain can change which backend it references."""
def test_swap_backend(self, temp_data_dir):
"""update_domain allows changing backend field."""
cfg = {
"backends": {
"webui": {"label": "WebUI", "paths": {}},
"myapp": {"label": "App", "paths": {}},
},
"domains": {"example.com": {"backend": "webui", "force_ssl": True}},
"ssl": {"protocols": "TLSv1.2 TLSv1.3"},
}
nginx.save_config(cfg)
nginx.update_domain("example.com", backend="myapp")
loaded = nginx.get_config()
assert loaded["domains"]["example.com"]["backend"] == "myapp"
def test_swap_backend_nonexistent_raises(self, temp_data_dir):
"""update_domain raises ValueError when new backend not found."""
cfg = {
"backends": {"webui": {"label": "WebUI", "paths": {}}},
"domains": {"example.com": {"backend": "webui"}},
"ssl": {"protocols": "TLSv1.2 TLSv1.3"},
}
nginx.save_config(cfg)
with pytest.raises(ValueError, match="Backend 'nonexistent' not found"):
nginx.update_domain("example.com", backend="nonexistent")
class TestGetDomainsWithBackends:
"""get_domains resolves paths from backends, includes backend_name."""
def test_flattens_by_backend_paths(self, temp_data_dir):
"""Each backend path becomes a separate entry."""
cfg = {
"backends": {
"myapp": {
"label": "My App",
"paths": {
"/": {"backend": {"host": "a", "port": 80, "proto": "http"}},
"/api": {
"backend": {"host": "b", "port": 8080, "proto": "http"}
},
"/ws": {
"backend": {"host": "c", "port": 9091, "proto": "http"},
"is_websocket": True,
},
},
}
},
"domains": {"example.com": {"backend": "myapp", "force_ssl": True}},
"ssl": {},
}
nginx.save_config(cfg)
result = nginx.get_domains()
assert len(result) == 3
paths = {r["path"] for r in result}
assert paths == {"/", "/api", "/ws"}
for r in result:
assert r["backend_name"] == "myapp"
assert r["domain"] == "example.com"
def test_skips_domains_without_backend(self, temp_data_dir):
"""Domains without a backend key are skipped."""
cfg = {
"backends": {},
"domains": {
"good.com": {"backend": "webui"},
"bad.com": {"some": "orphan"},
},
"ssl": {},
}
nginx.save_config(cfg)
result = nginx.get_domains()
domains = {r["domain"] for r in result}
assert "good.com" in domains
assert "bad.com" not in domains
def test_includes_management_and_websocket_flags(self, temp_data_dir):
"""is_management and is_websocket flags propagate to entries."""
cfg = {
"backends": {
"webui": {
"label": "WebUI",
"paths": {
"/": {
"backend": {
"host": "127.0.0.1",
"port": 9090,
"proto": "http",
},
"is_management": True,
},
"/ws": {
"backend": {
"host": "127.0.0.1",
"port": 9091,
"proto": "http",
},
"is_websocket": True,
},
},
}
},
"domains": {"mgmt.local": {"backend": "webui"}},
"ssl": {},
}
nginx.save_config(cfg)
result = nginx.get_domains()
entries_by_path = {r["path"]: r for r in result}
assert entries_by_path["/"]["is_management"] is True
assert entries_by_path["/ws"]["is_websocket"] is True
class TestGenerateServerConfWithBackends:
"""generate_server_conf resolves paths via backends parameter."""
def test_with_backends_param(self, temp_data_dir):
"""When backends provided, paths resolve from backend."""
backends = {
"myapp": {
"paths": {
"/": {
"backend": {"host": "10.0.0.1", "port": 8080, "proto": "http"}
}
}
}
}
domain_cfg = {"domain": "example.com", "backend": "myapp", "force_ssl": True}
out = nginx.generate_server_conf(domain_cfg, backends)
assert "proxy_pass http://10.0.0.1:8080;" in out
class TestPatchConfigPreservesBackends:
"""PATCH_NGINX_CONFIG (deep_merge) does not overwrite backends fully."""
def test_ssl_patch_keeps_backends(self, temp_data_dir):
"""Patching ssl settings preserves backends dict (deep_merge semantics)."""
from lib.common import deep_merge
current = {
"backends": {"myapp": {"label": "App", "paths": {}}},
"domains": {"example.com": {"backend": "myapp"}},
"ssl": {"protocols": "TLSv1.2"},
}
patch = {"ssl": {"protocols": "TLSv1.3"}}
merged = deep_merge(current, patch)
assert merged["backends"]["myapp"]["label"] == "App"
assert merged["domains"]["example.com"]["backend"] == "myapp"
assert merged["ssl"]["protocols"] == "TLSv1.3"
+17 -9
View File
@@ -14,6 +14,11 @@ class TestPollIntervals:
assert _POLL_INTERVALS["wireguard"] == 10
assert _POLL_INTERVALS["dnsmasq"] == 10
assert _POLL_INTERVALS["networkd"] == 10
# Phase 5: real-time system metrics poll at 1s.
assert _POLL_INTERVALS["system"] == 1
# nginx/acme derive from config files; poll for drift self-heal.
assert _POLL_INTERVALS["nginx"] == 60
assert _POLL_INTERVALS["acme"] == 300
def test_env_override(self):
"""VACUUM_WALL_POLL_INTERVALS env var can override values."""
@@ -33,28 +38,31 @@ class TestPollIntervals:
class TestBroadcastTick:
def test_sends_tick_message(self):
from daemon.server import _ws_subscribers, broadcast_tick
import daemon.server as server
mock_ws = AsyncMock()
mock_ws.send_str = AsyncMock()
_ws_subscribers.add(mock_ws)
server._ws_subscribers.add(mock_ws)
try:
asyncio.run(broadcast_tick(["firewall", "wireguard"]))
with patch.object(server.state_store, "get", return_value={"up": True}):
asyncio.run(server.broadcast_tick("firewall"))
mock_ws.send_str.assert_called_once()
sent = json.loads(mock_ws.send_str.call_args[0][0])
assert sent["type"] == "tick"
assert sent["subsystems"] == ["firewall", "wireguard"]
assert sent["subsystem"] == "firewall"
assert sent["data"] == {"up": True}
finally:
_ws_subscribers.discard(mock_ws)
server._ws_subscribers.discard(mock_ws)
def test_prunes_dead_subscribers(self):
from daemon.server import _ws_subscribers, broadcast_tick
import daemon.server as server
mock_ws = AsyncMock()
mock_ws.send_str = AsyncMock(side_effect=Exception("broken"))
_ws_subscribers.add(mock_ws)
asyncio.run(broadcast_tick(["firewall"]))
assert mock_ws not in _ws_subscribers
server._ws_subscribers.add(mock_ws)
with patch.object(server.state_store, "get", return_value={"up": True}):
asyncio.run(server.broadcast_tick("firewall"))
assert mock_ws not in server._ws_subscribers
class TestPollTasks:
+144
View File
@@ -0,0 +1,144 @@
"""Tests for refresh_state / refresh_status WS broadcasting (daemon.server).
refresh_state() and refresh_status() re-collect state and broadcast a
data-carrying versions message for every (requested) subsystem so all
viewers stay in sync.
"""
import asyncio
import json
from contextlib import suppress
from unittest.mock import AsyncMock, MagicMock, patch
import daemon.server as server
def _run_and_drain(fn):
"""Run *fn* inside a running event loop (required for broadcast tasks),
then drain the fire-and-forget broadcast tasks."""
async def drive():
fn()
for _ in range(20):
await asyncio.sleep(0)
if not server._ws_tasks:
break
for task in list(server._ws_tasks):
with suppress(Exception):
await task
asyncio.run(drive())
def _new_ws():
ws = AsyncMock()
ws.send_str = AsyncMock()
server._ws_subscribers.add(ws)
return ws
def _messages(ws):
return [json.loads(c[0][0]) for c in ws.send_str.call_args_list]
class TestRefreshStateBroadcast:
def test_broadcasts_each_requested_subsystem(self):
"""refresh_state(["firewall","dnsmasq"]) broadcasts both, and bumps."""
store = MagicMock()
store.get.side_effect = lambda name: {"s": name}
ws = _new_ws()
try:
with patch.object(server, "state_store", store):
_run_and_drain(lambda: server.refresh_state(["firewall", "dnsmasq"]))
store.populate.assert_called_once_with(["firewall", "dnsmasq"])
store.bump.assert_any_call("firewall")
store.bump.assert_any_call("dnsmasq")
msgs = _messages(ws)
assert sorted(m["subsystem"] for m in msgs) == ["dnsmasq", "firewall"]
assert all(m["type"] == "versions" for m in msgs)
for m in msgs:
assert "updated" not in m
assert m["data"] == {"s": m["subsystem"]}
finally:
server._ws_subscribers.discard(ws)
def test_failed_subsystem_skipped_others_buzz(self):
"""A subsystem whose collection failed (None) is not broadcast."""
store = MagicMock()
store.get.side_effect = lambda name: {"s": name} if name != "acme" else None
ws = _new_ws()
try:
with patch.object(server, "state_store", store):
_run_and_drain(lambda: server.refresh_state(["firewall", "acme"]))
subs = sorted(m["subsystem"] for m in _messages(ws))
assert subs == ["firewall"]
finally:
server._ws_subscribers.discard(ws)
def test_no_subsystems_arg_broadcasts_all(self):
"""refresh_state() with no filter targets every subsystem."""
from lib.state import State
store = State()
for name in State.SUBSYSTEMS:
store.set(name, {"k": name})
ws = _new_ws()
try:
with patch.object(server, "state_store", store):
_run_and_drain(lambda: server.refresh_state())
subs = sorted(m["subsystem"] for m in _messages(ws))
assert subs == sorted(State.SUBSYSTEMS)
finally:
server._ws_subscribers.discard(ws)
class TestRefreshStatusBroadcast:
def test_filtered_response_and_broadcast(self):
"""POST /status/refresh replies only with the requested subsystems
and broadcasts each of them."""
store = MagicMock()
store.get.side_effect = lambda name: {"s": name}
ws = _new_ws()
try:
with patch.object(server, "state_store", store):
async def drive():
request = MagicMock()
request.json = AsyncMock(return_value={"subsystems": ["firewall"]})
response = await server.refresh_status(request)
await asyncio.sleep(0.01)
return response
response = asyncio.run(drive())
body = json.loads(response.body)
assert body["ok"] is True
assert set(body["data"]) == {"firewall"}
store.bump.assert_not_called()
subs = sorted(m["subsystem"] for m in _messages(ws))
assert subs == ["firewall"]
finally:
server._ws_subscribers.discard(ws)
def test_no_body_returns_all_subsystems(self):
store = MagicMock()
store.get.side_effect = lambda name: {"s": name}
store.SUBSYSTEMS = ["firewall", "dnsmasq"]
ws = _new_ws()
try:
with patch.object(server, "state_store", store):
async def drive():
request = MagicMock()
request.json = AsyncMock(return_value=None)
response = await server.refresh_status(request)
await asyncio.sleep(0.01)
return response
response = asyncio.run(drive())
body = json.loads(response.body)
assert body["ok"] is True
assert set(body["data"]) == {"firewall", "dnsmasq"}
finally:
server._ws_subscribers.discard(ws)
+175
View File
@@ -0,0 +1,175 @@
"""Tests that collector outputs match the lib.schema TypedDict shapes.
Each collector's return value is asserted against its TypedDict's required
keys at runtime (subprocess/shell calls mocked no system services). The
TypedDicts in lib/schema.py are the authoritative state-store contract;
these tests catch drift between the schemas and the collectors.
"""
import json
from unittest.mock import Mock, patch
import daemon.collectors.acme
import daemon.collectors.dnsmasq
import daemon.collectors.firewall
import daemon.collectors.networkd
import daemon.collectors.nginx
import daemon.collectors.system
import daemon.collectors.wireguard
from lib import schema
def _missing(required_keys: frozenset, data: dict) -> set[str]:
return set(required_keys) - set(data)
class TestCollectorShapesMatchSchema:
def test_firewall_state(self):
with (
patch.object(daemon.collectors.firewall, "run") as mock_run,
patch.object(
daemon.collectors.firewall,
"_network_get_config",
return_value={
"interfaces": {
"eth0": {},
"eth1": {},
"lo": {},
"wg0": {},
}
},
),
):
def run_side(args, **kwargs):
if "--get-active-zones" in args:
return "public\n eth0"
if "--get-services" in args:
return "ssh http"
if "--list-all-zones" in args:
return (
"public\n"
" target: default\n"
" interfaces: eth0\n"
" services: \n"
" ports: \n"
" protocols: \n"
" forward-ports: \n"
" masquerade: no\n"
" rich rules: \n"
)
if "ip" in args[0]:
if "link" in args:
return (
"1: lo: <LOOPBACK,UP> mtu 65536\n"
"2: eth0: <BROADCAST,UP> mtu 1500 link/ether aa:bb\n"
)
if "addr" in args:
return "2: eth0 inet 192.168.1.1/24\n"
return ""
mock_run.side_effect = run_side
result = daemon.collectors.firewall._collect_firewall()
assert not _missing(schema.FirewallState.__required_keys__, result)
for iface in result["interfaces"]:
for k in schema.FirewallInterface.__required_keys__:
assert k in iface, f"FirewallInterface missing {k}"
# eth1 is network-config-managed but in no live zone; lo and wg*
# are filtered out even though they are present in the config.
assert result["uncovered_interfaces"] == ["eth1"]
def test_dnsmasq_state(self):
with patch.object(daemon.collectors.dnsmasq, "run_proc") as mock_proc:
mock_proc.return_value = Mock(stdout="active\n", returncode=0)
result = daemon.collectors.dnsmasq._collect_dnsmasq()
assert not _missing(schema.DnsmasqState.__required_keys__, result)
for k in schema.DnsmasqStatus.__required_keys__:
assert k in result["status"], f"DnsmasqStatus missing {k}"
def test_nginx_state(self):
result = daemon.collectors.nginx._collect_nginx()
assert not _missing(schema.NginxState.__required_keys__, result)
assert "pending_changes" in result["status"]
def test_acme_state(self):
with (
patch.object(
daemon.collectors.acme, "_get_acme_email", return_value="a@b.c"
),
patch("lib.acme.list_certs", return_value=[]),
patch.object(
daemon.collectors.acme,
"_parse_account_conf",
return_value={"registered": False, "email": "", "ca": ""},
),
):
result = daemon.collectors.acme._collect_acme()
assert not _missing(schema.AcmeState.__required_keys__, result)
assert result["status"]["error"] is None
def test_wireguard_state(self):
with patch.object(daemon.collectors.wireguard, "run_proc") as mock_proc:
mock_proc.return_value = Mock(stdout="", returncode=1)
result = daemon.collectors.wireguard._collect_wireguard()
assert not _missing(schema.WgState.__required_keys__, result)
for k in schema.WgStatus.__required_keys__:
assert k in result["status"], f"WgStatus missing {k}"
assert "classes" in result["status"]
def test_networkd_state(self):
networkctl = {
"Interfaces": [
{
"Name": "eth0",
"Type": "ether",
"OperationalState": "routable",
"HardwareAddress": [1, 2, 3, 4, 5, 6],
"Addresses": [
{"Address": [192, 168, 30, 50], "Family": 2, "PrefixLength": 24}
],
"Routes": [
{
"Family": 2,
"Destination": [0, 0, 0, 0],
"DestinationPrefixLength": 0,
"Gateway": [192, 168, 30, 1],
}
],
"DNS": [{"Address": [1, 1, 1, 1], "Family": 2}],
}
]
}
with patch.object(
daemon.collectors.networkd, "run", return_value=json.dumps(networkctl)
):
result = daemon.collectors.networkd._collect_networkd()
assert not _missing(schema.NetworkdState.__required_keys__, result)
assert "eth0" in result["interfaces"]
entry = result["interfaces"]["eth0"]
for k in schema.NetworkdInterface.__required_keys__:
assert k in entry, f"NetworkdInterface missing {k}"
assert entry["gateway"] == "192.168.30.1"
assert entry["addresses"] == ["192.168.30.50/24"]
def test_system_state(self):
"""Reads /proc and /sys directly — no mocking needed on Linux."""
result = daemon.collectors.system._collect_system()
assert not _missing(schema.SystemState.__required_keys__, result)
for k in schema.CpuLoad.__required_keys__:
assert k in result["load"], f"CpuLoad missing {k}"
for k in schema.MemoryStats.__required_keys__:
assert k in result["memory"], f"MemoryStats missing {k}"
for k in schema.SwapStats.__required_keys__:
assert k in result["swap"], f"SwapStats missing {k}"
def test_volatile_system_registered(self):
"""Phase 5: system metrics are volatile (tick, not version bumps)."""
from lib.state import _VOLATILE
expected = frozenset({"load", "memory", "swap", "traffic"})
assert _VOLATILE.get("system") == expected
+55 -27
View File
@@ -4,6 +4,8 @@ from unittest.mock import patch
import pytest
from webui.server import _has_permission, _subsystem_from_path
@pytest.fixture
def client():
@@ -30,44 +32,33 @@ class TestSPARoutes:
def test_api_routes_still_work(self, client):
resp = client.get("/api/firewall/zones")
assert resp.status_code in (200, 500)
assert resp.status_code in (401, 500)
data = resp.get_json()
assert data is not None
assert data.get("ok") is False
assert data.get("error") == "unauthorized"
class TestWsUrlGeneration:
def test_ws_url_ipv4_host(self, client):
resp = client.get("/", headers={"Host": "192.168.1.1:9090"})
assert b"ws://192.168.1.1:9090/ws" in resp.data
def test_ws_url_ipv6_host(self, client):
resp = client.get("/", headers={"Host": "[::1]:9090"})
assert b"ws://[::1]:9090/ws" in resp.data
class TestApiStatusAll:
@patch("webui.server.get")
def test_success(self, mock_get, client):
mock_get.return_value = {"firewall": {"zones": {}}, "dnsmasq": {}}
resp = client.get("/api/status/all")
class TestSpaRoot:
def test_serves_index_html_as_is(self, client):
resp = client.get("/")
assert resp.status_code == 200
data = resp.get_json()
assert data["ok"] is True
assert "firewall" in data["data"]
assert b"/static/app.js" in resp.data
@patch("webui.server.get")
def test_error(self, mock_get, client):
mock_get.side_effect = RuntimeError("connection refused")
resp = client.get("/api/status/all")
assert resp.status_code == 500
data = resp.get_json()
assert data["ok"] is False
def test_no_ws_url_substitution(self, client):
"""index.html is served verbatim — no WS URL placeholder substitution."""
resp = client.get("/", headers={"Host": "192.168.1.1:9090"})
assert b"__WS_URL_PLACEHOLDER__" not in resp.data
assert b"ws://" not in resp.data
class TestBlueprintsRegistered:
def test_all_blueprints_registered(self, client):
from webui.server import BLUEPRINTS
assert len(BLUEPRINTS) == 7
assert len(BLUEPRINTS) == 9
names = [name for name, _ in BLUEPRINTS]
assert "auth" in names
assert "firewall" in names
assert "network" in names
assert "dhcp" in names
@@ -109,3 +100,40 @@ class TestGroupWriteHandler:
mode = os.stat(log_file).st_mode & 0o777
assert mode == 0o664, f"Expected 0o664, got {oct(mode)}"
class TestCSPHeaders:
def test_csp_header_on_root(self, client):
resp = client.get("/")
csp = resp.headers.get("Content-Security-Policy")
assert csp is not None
assert "default-src 'self'" in csp
assert "script-src 'self'" in csp
assert "'unsafe-inline'" not in csp.split("script-src")[1].split(";")[0]
def test_x_content_type_options(self, client):
resp = client.get("/")
assert resp.headers.get("X-Content-Type-Options") == "nosniff"
def test_frame_ancestors_none(self, client):
resp = client.get("/")
csp = resp.headers.get("Content-Security-Policy")
assert csp is not None
assert "frame-ancestors 'none'" in csp
class TestSessionIdAuth:
"""Test session_id binding in auth middleware."""
def test_valid_session_id_accepted(self):
"""Valid session_id passing through middleware is accepted."""
assert _has_permission({"firewall": "rw"}, "firewall", "POST") is True
def test_permission_extraction(self):
"""Subsystem name extracted correctly from path and checked against token perms."""
sub = _subsystem_from_path("/api/firewall/zones")
assert sub == "firewall"
perms = {"firewall": "read"}
assert _has_permission(perms, "firewall", "GET") is True
assert _has_permission(perms, "firewall", "POST") is False
+362 -20
View File
@@ -1,7 +1,12 @@
"""Tests for lib/state.py — state store and collect functions."""
import json
import os
from unittest.mock import patch
import daemon.collectors.acme
import daemon.collectors.dnsmasq
import daemon.collectors.firewall
from lib.state import State, state
@@ -28,44 +33,74 @@ class TestState:
assert state is not None
assert isinstance(state, State)
def test_get_snapshot_empty(self):
"""Fresh store: snapshot lists every subsystem, all None."""
s = State()
snap = s.get_snapshot()
assert set(snap) == set(s.SUBSYSTEMS)
assert all(v is None for v in snap.values())
def test_get_snapshot_reflects_set_and_none(self):
"""Snapshot carries set data; failed collections stay None."""
s = State()
s.set("firewall", {"zones": {}})
s.set("system", {"load": {"load1": 0.0}})
s.set("dnsmasq", None)
snap = s.get_snapshot()
assert snap["firewall"] == {"zones": {}}
assert snap["system"] == {"load": {"load1": 0.0}}
assert snap["dnsmasq"] is None
assert snap["acme"] is None
class TestCollectAll:
@patch("lib.state.run")
@patch("daemon.collectors.firewall.run")
def test_collect_firewall_returns_dict(self, mock_run):
from lib.state import _collect_firewall
from daemon.collectors.firewall import _collect_firewall
def run_side(args, **kwargs):
if "--get-zones" in args:
return "public\ninternal"
if "--get-active-zones" in args:
return "public\n eth0"
if "--get-default-zone" in args:
return "public\n"
if "--get-services" in args:
return "ssh http"
if "ip" in args[0]:
if "link" in args:
return "1: lo: <LOOPBACK> mtu 65536\n2: eth0: <UP> mtu 1500 link/ether aa:bb\n"
return ""
if "--list-all" in args:
return "target: default\ninterfaces: eth0\nsources: \nservices: \nports: \nprotocols: \nforward-ports: \nmasquerade: no\nics: no\nrich-rules: \nicmp-blocks: \nmodule: \n"
return ""
if "--list-all-zones" in args:
return (
"public\n"
" target: default\n"
" interfaces: eth0\n"
" services: \n"
" ports: \n"
" protocols: \n"
" forward-ports: \n"
" masquerade: no\n"
" rich rules: \n"
)
mock_run.side_effect = run_side
result = _collect_firewall()
assert isinstance(result, dict)
assert "active_zones" in result
assert "default_zone" in result
assert result["default_zone"] == "public"
assert "interfaces" in result
assert "timestamp" in result
@patch("lib.state.run")
@patch("daemon.collectors.firewall.run")
def test_collect_firewall_vlan_ips_populated(self, mock_run):
"""VLAN interfaces with @suffix in ip addr output get their IPs collected."""
from lib.state import _collect_firewall
from daemon.collectors.firewall import _collect_firewall
def run_side(args, **kwargs):
if "--get-zones" in args:
return "public\ninternal"
if "--get-active-zones" in args:
return "public\n eth0\ninternal eth0.100"
return "public\n eth0\ninternal\n eth0.100"
if "--get-default-zone" in args:
return "public\n"
if "--get-services" in args:
return "ssh http"
if "ip" in args[0]:
@@ -81,14 +116,27 @@ class TestCollectAll:
"3: eth0.100@if100 inet 10.0.0.1/24\n"
)
return ""
if "--list-all" in args:
if "--list-all-zones" in args:
return (
"target: default\ninterfaces: eth0\nsources: "
"services: \nports: \nprotocols: \nforward-ports: "
"masquerade: no\nics: no\nrich-rules: "
"icmp-blocks: \nmodule: \n"
"public\n"
" target: default\n"
" interfaces: eth0\n"
" services: \n"
" ports: \n"
" protocols: \n"
" forward-ports: \n"
" masquerade: no\n"
" rich rules: \n"
"internal\n"
" target: ACCEPT\n"
" interfaces: eth0.100\n"
" services: \n"
" ports: \n"
" protocols: \n"
" forward-ports: \n"
" masquerade: no\n"
" rich rules: \n"
)
return ""
mock_run.side_effect = run_side
result = _collect_firewall()
@@ -99,11 +147,35 @@ class TestCollectAll:
assert vlan_iface["ips"], "VLAN interface should have collected IPs"
assert "10.0.0.1/24" in vlan_iface["ips"]
@patch("lib.state.run_proc")
@patch("daemon.collectors.firewall.get_service_descriptions")
@patch("daemon.collectors.firewall.run")
def test_collect_firewall_includes_service_descriptions(self, mock_run, mock_desc):
from daemon.collectors.firewall import _collect_firewall
def run_side(args, **kwargs):
if "--get-active-zones" in args:
return ""
if "--get-default-zone" in args:
return "public\n"
if "--get-services" in args:
return "ssh http"
if "ip" in args[0]:
return ""
if "--list-all-zones" in args:
return ""
mock_run.side_effect = run_side
descs = {"ssh": "OpenSSH", "http": "WWW"}
mock_desc.return_value = descs
result = _collect_firewall()
mock_desc.assert_called_once_with()
assert result["service_descriptions"] == descs
@patch("daemon.collectors.dnsmasq.run_proc")
def test_collect_dnsmasq_returns_dict(self, mock_proc):
from unittest.mock import Mock
from lib.state import _collect_dnsmasq
from daemon.collectors.dnsmasq import _collect_dnsmasq
mock_proc.return_value = Mock(stdout="active\n", returncode=0)
result = _collect_dnsmasq()
@@ -112,6 +184,66 @@ class TestCollectAll:
assert "config" in result
assert "leases" in result
@patch("daemon.collectors.dnsmasq.run_proc")
def test_collect_dnsmasq_pending_diff(self, mock_proc, tmp_path, monkeypatch):
from unittest.mock import Mock
from daemon.collectors.dnsmasq import _collect_dnsmasq
from lib.common import _APPLY_HASH_KEY, _LAST_APPLIED_CONFIG_KEY
(tmp_path / "config" / "dnsmasq").mkdir(parents=True)
applied = {
"dhcp": {
"ranges": [
{
"interface": "eth1",
"start": "10.4.20.101",
"end": "10.4.20.200",
"lease_time": "1h",
"gateway": "10.4.20.1",
}
],
"static_leases": [],
},
"dns": {"upstreams": ["8.8.8.8"], "domain": None, "custom_records": []},
}
cfg = {
"dhcp": {
"ranges": [
{
"interface": "eth1",
"start": "10.4.20.100",
"end": "10.4.20.200",
"lease_time": "12h",
"gateway": "10.4.20.1",
}
],
"static_leases": [],
},
"dns": {
"upstreams": ["8.8.8.8", "1.1.1.1"],
"domain": None,
"custom_records": [],
},
_LAST_APPLIED_CONFIG_KEY: applied,
_APPLY_HASH_KEY: "stale-hash",
}
(tmp_path / "config" / "dnsmasq" / "config.json").write_text(json.dumps(cfg))
monkeypatch.setattr(
"lib.dnsmasq.CONFIG_PATH", tmp_path / "config" / "dnsmasq" / "config.json"
)
# service check -> active; lease file read -> no lines
mock_proc.return_value = Mock(stdout="active\n", returncode=0)
result = _collect_dnsmasq()
assert result["status"]["pending_changes"] is True
paths = {d["path"] for d in result["status"]["pending_diff"]}
assert "dhcp.ranges[0].start" in paths
assert "dhcp.ranges[0].lease_time" in paths
# Apply metadata must not leak into the returned config.
assert _LAST_APPLIED_CONFIG_KEY not in result["config"]
assert _APPLY_HASH_KEY not in result["config"]
class TestCollectFailure:
def test_state_clears_on_failure(self):
@@ -123,6 +255,216 @@ class TestCollectFailure:
assert s.is_populated() is False
_ACCOUNT = {"registered": False, "email": "", "ca": ""}
class TestAcmeCollectNonFatal:
"""A broken acme.sh must not clear the acme subsystem (dashboard guard)."""
def test_list_failure_yields_empty_certs_and_error(self):
from daemon.collectors.acme import _collect_acme
with (
patch.object(
daemon.collectors.acme, "_get_acme_email", return_value="a@b.c"
),
patch(
"daemon.collectors.acme._acme_home_needs_normalize", return_value=True
),
patch("daemon.handlers.acme.normalize_acme_home"),
patch(
"lib.acme.list_certs",
side_effect=RuntimeError("acme.sh failed with exit code 2"),
),
patch.object(
daemon.collectors.acme, "_parse_account_conf", return_value=_ACCOUNT
),
):
result = _collect_acme()
assert result["certs"] == []
assert result["email"] == "a@b.c"
assert result["status"]["error"] is not None
assert "exit code 2" in result["status"]["error"]
def test_success_reports_no_error(self):
from daemon.collectors.acme import _collect_acme
with (
patch.object(
daemon.collectors.acme, "_get_acme_email", return_value="a@b.c"
),
patch(
"daemon.collectors.acme._acme_home_needs_normalize", return_value=True
),
patch("daemon.handlers.acme.normalize_acme_home"),
patch("lib.acme.list_certs", return_value=[]),
patch.object(
daemon.collectors.acme, "_parse_account_conf", return_value=_ACCOUNT
),
):
result = _collect_acme()
assert result["status"] == {"error": None}
def test_self_heal_normalizes_before_list(self):
from daemon.collectors.acme import _collect_acme
order: list[str] = []
def _norm():
order.append("normalize")
def _list():
order.append("list")
return [{"domain": "example.com"}]
with (
patch.object(
daemon.collectors.acme, "_get_acme_email", return_value="a@b.c"
),
patch(
"daemon.collectors.acme._acme_home_needs_normalize", return_value=True
),
patch("daemon.handlers.acme.normalize_acme_home", side_effect=_norm),
patch("lib.acme.list_certs", side_effect=_list),
patch.object(
daemon.collectors.acme, "_parse_account_conf", return_value=_ACCOUNT
),
):
result = _collect_acme()
# The poll must normalize ACME_HOME perms before listing when the
# probe detects a lost group-read bit, so a mid-lifetime ownership
# flip self-heals without a restart.
assert order == ["normalize", "list"]
assert result["certs"] == [{"domain": "example.com"}]
assert result["status"] == {"error": None}
def test_no_normalize_when_probe_clean(self):
from daemon.collectors.acme import _collect_acme
order: list[str] = []
def _norm():
order.append("normalize")
def _list():
order.append("list")
return []
with (
patch.object(
daemon.collectors.acme, "_get_acme_email", return_value="a@b.c"
),
patch(
"daemon.collectors.acme._acme_home_needs_normalize", return_value=False
),
patch("daemon.handlers.acme.normalize_acme_home", side_effect=_norm),
patch("lib.acme.list_certs", side_effect=_list),
patch.object(
daemon.collectors.acme, "_parse_account_conf", return_value=_ACCOUNT
),
):
result = _collect_acme()
# Steady state: the probe sees group-read bits intact, so the poll
# must not pay for a sudo normalize.
assert order == ["list"]
assert result["certs"] == []
assert result["status"] == {"error": None}
class TestAcmeHomeProbe:
"""_acme_home_needs_normalize probes the group-read bit without sudo."""
def test_flags_file_without_group_read(self, tmp_path, monkeypatch):
from daemon.collectors.acme import _acme_home_needs_normalize
monkeypatch.setenv("ACME_HOME", str(tmp_path))
(tmp_path / "account.conf").write_text("x")
os.chmod(tmp_path / "account.conf", 0o600)
assert _acme_home_needs_normalize() is True
def test_clean_when_group_read_set(self, tmp_path, monkeypatch):
from daemon.collectors.acme import _acme_home_needs_normalize
monkeypatch.setenv("ACME_HOME", str(tmp_path))
(tmp_path / "account.conf").write_text("x")
os.chmod(tmp_path / "account.conf", 0o640)
assert _acme_home_needs_normalize() is False
def test_clean_on_empty_home(self, tmp_path, monkeypatch):
from daemon.collectors.acme import _acme_home_needs_normalize
monkeypatch.setenv("ACME_HOME", str(tmp_path))
assert _acme_home_needs_normalize() is False
def test_clean_on_missing_home(self, tmp_path, monkeypatch):
from daemon.collectors.acme import _acme_home_needs_normalize
monkeypatch.setenv("ACME_HOME", str(tmp_path / "does-not-exist"))
assert _acme_home_needs_normalize() is False
def test_permission_error_is_actionable(self):
from daemon.collectors.acme import _collect_acme
msg = "acme.sh failed with exit code 2: .../account.conf: Permission denied"
with (
patch.object(
daemon.collectors.acme, "_get_acme_email", return_value="a@b.c"
),
patch("daemon.handlers.acme.normalize_acme_home"),
patch("lib.acme.list_certs", side_effect=RuntimeError(msg)),
patch.object(
daemon.collectors.acme, "_parse_account_conf", return_value=_ACCOUNT
),
):
result = _collect_acme()
assert result["status"]["error"] is not None
assert "sudo chown" in result["status"]["error"]
class TestParseAccountConf:
"""_parse_account_conf reads acme.sh v3's account.conf (no leading dot)."""
def test_reads_no_dot_account_conf(self, tmp_path):
from daemon.collectors.acme import _parse_account_conf
(tmp_path / "account.conf").write_text(
"ACME_LEEMAIL='me@example.com'\nACME_MCA='zerossl'\nACME_CERTKEYSIZE=256\n"
)
acct = _parse_account_conf(acme_home=tmp_path)
assert acct["registered"] is True
assert acct["email"] == "me@example.com"
assert acct["ca"] == "ZeroSSL"
assert acct["key_length"] == 256
def test_prefers_no_dot_over_legacy_dot(self, tmp_path):
from daemon.collectors.acme import _parse_account_conf
(tmp_path / "account.conf").write_text(
"ACME_LEEMAIL='new@example.com'\nACME_MCA='letsencrypt'\n"
)
(tmp_path / ".account.conf").write_text(
"ACME_LEEMAIL='old@example.com'\nACME_MCA='zerossl'\n"
)
acct = _parse_account_conf(acme_home=tmp_path)
assert acct["email"] == "new@example.com"
def test_falls_back_to_legacy_dot(self, tmp_path):
from daemon.collectors.acme import _parse_account_conf
(tmp_path / ".account.conf").write_text(
"ACME_LEEMAIL='legacy@example.com'\nACME_MCA='zerossl'\n"
)
acct = _parse_account_conf(acme_home=tmp_path)
assert acct["registered"] is True
assert acct["email"] == "legacy@example.com"
assert acct["ca"] == "ZeroSSL"
class TestStateVersions:
def test_version_starts_at_zero(self):
s = State()
+221
View File
@@ -0,0 +1,221 @@
"""Tests for daemon/handlers/status.py — cancel-all (revert to applied)."""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any, ClassVar
from unittest.mock import patch
from daemon.handlers import status
from lib.common import (
_APPLY_HASH_KEY,
_LAST_APPLIED_CONFIG_KEY,
config_hash,
stamp_applied,
)
def _pending(firewall: bool = False, **flags: bool) -> dict[str, Any]:
"""Build status_pending() output with the given subsystems pending.
Mirrors the real handler shape: every subsystem key is always present.
"""
data: dict[str, Any] = {
"firewall": {
"needs_apply": firewall,
"change_count": 1 if firewall else 0,
"changes": [],
},
}
total = 1 if firewall else 0
for name in ("dnsmasq", "nginx", "wireguard", "networkd"):
flagged = bool(flags.get(name, False))
data[name] = {
"pending_changes": flagged,
"summary": "x" if flagged else "Up to date",
"changes": [],
}
total += 1 if flagged else 0
data["total_changes"] = total
return data
class TestConfigPathResolution:
"""SYS_CONFIG_PATHS points at the constants the handlers actually use."""
def test_resolves_to_module_constants(self):
import daemon.handlers.dnsmasq as dm_h
import daemon.handlers.firewall as fw_h
import daemon.handlers.nginx as ngx_h
import lib.network as net_lib
import lib.wireguard as wg_lib
assert status._config_path("firewall") == fw_h.CONFIG_FILE
assert status._config_path("dnsmasq") == dm_h.CONFIG_PATH
assert status._config_path("nginx") == ngx_h.CONFIG_FILE
assert status._config_path("wireguard") == wg_lib.CONFIG_PATH
assert status._config_path("networkd") == net_lib.CONFIG_FILE
def test_all_subsystems_mapped(self):
assert set(status.SYS_CONFIG_PATHS) == set(status.SYS_ORDER)
class TestStatusCancelAll:
"""Test the cancel-all endpoint.
Patches the ``_config_path`` seam directly since it resolves the real
module constants via getattr at call time.
"""
_fake_pending_all: ClassVar[dict[str, Any]] = _pending()
def _write_config(
self, path: Path, cfg: dict[str, Any], baseline: dict | None
) -> None:
"""Write *cfg* (with an attached applied baseline when *baseline* given)."""
path.parent.mkdir(parents=True, exist_ok=True)
stamped = dict(cfg)
if baseline is not None:
b = dict(baseline)
stamp_applied(b)
stamped[_LAST_APPLIED_CONFIG_KEY] = dict(baseline)
stamped[_APPLY_HASH_KEY] = b[_APPLY_HASH_KEY]
path.write_text(json.dumps(stamped, indent=2) + "\n")
def _read_config(self, path: Path) -> dict[str, Any]:
return json.loads(path.read_text())
@patch("daemon.handlers.status.status_pending")
@patch("daemon.handlers.status.refresh_state")
def test_nothing_pending(self, mock_refresh, mock_pending):
mock_pending.return_value = self._fake_pending_all
with patch("daemon.handlers.status._config_path", side_effect=AssertionError):
result = status.status_cancel_all(None, None)
assert result == {"cancelled": [], "skipped": {}, "errors": {}}
mock_refresh.assert_not_called()
def test_reverts_pending_subsystem(self, tmp_path):
paths = {name: tmp_path / name / "config.json" for name in status.SYS_ORDER}
applied = {"dns": {"upstreams": ["8.8.8.8"]}}
# Baseline applied, then the user edited the config (drift).
self._write_config(
paths["dnsmasq"], {"dns": {"upstreams": ["1.1.1.1"]}}, applied
)
dirty = self._read_config(paths["dnsmasq"])
assert dirty[_APPLY_HASH_KEY] != config_hash(dirty) # really pending
with (
patch(
"daemon.handlers.status.status_pending",
return_value=_pending(dnsmasq=True),
),
patch("daemon.handlers.status.refresh_state") as mock_refresh,
patch(
"daemon.handlers.status._config_path",
side_effect=lambda name: paths[name],
),
):
result = status.status_cancel_all(None, None)
assert result == {
"cancelled": ["dnsmasq"],
"skipped": {},
"errors": {},
}
mock_refresh.assert_called_once_with(status.SYS_ORDER)
restored = self._read_config(paths["dnsmasq"])
assert restored.get(_LAST_APPLIED_CONFIG_KEY) == applied
# Cancel restored the baseline: the pending check now passes.
assert restored[_APPLY_HASH_KEY] == config_hash(restored)
def test_skipped_when_no_baseline(self, tmp_path):
paths = {name: tmp_path / name / "config.json" for name in status.SYS_ORDER}
cfg_no_baseline = {"peers": {"a": {}}}
self._write_config(paths["wireguard"], cfg_no_baseline, None)
with (
patch(
"daemon.handlers.status.status_pending",
return_value=_pending(wireguard=True),
),
patch("daemon.handlers.status.refresh_state") as mock_refresh,
patch(
"daemon.handlers.status._config_path",
side_effect=lambda name: paths[name],
),
):
result = status.status_cancel_all(None, None)
assert result["cancelled"] == []
assert "WireGuard" in result["skipped"]
assert result["errors"] == {}
mock_refresh.assert_not_called()
# File left untouched.
assert self._read_config(paths["wireguard"]) == cfg_no_baseline
def test_error_in_one_subsystem_others_still_cancel(self, tmp_path):
paths = {name: tmp_path / name / "config.json" for name in status.SYS_ORDER}
for name in ("firewall", "nginx"):
self._write_config(
paths[name],
{"zones": {}} if name == "firewall" else {"domains": {}},
{"x": 1},
)
def fake_revert(path):
if "nginx" in str(path):
raise RuntimeError("disk full")
return True, ""
with (
patch(
"daemon.handlers.status.status_pending",
return_value=_pending(firewall=True, nginx=True),
),
patch("daemon.handlers.status.refresh_state"),
patch("daemon.handlers.status.revert_to_applied", side_effect=fake_revert),
patch(
"daemon.handlers.status._config_path",
side_effect=lambda name: paths[name],
),
):
result = status.status_cancel_all(None, None)
# SYS_ORDER runs firewall before nginx.
assert result["cancelled"] == ["firewall"]
assert result["errors"] == {"Nginx": "disk full"}
assert result["skipped"] == {}
def test_order_matches_sys_order(self, tmp_path):
paths = {name: tmp_path / name / "config.json" for name in status.SYS_ORDER}
call_order = []
def fake_revert(path):
call_order.append(next(n for n, p in paths.items() if p == path))
return True, ""
with (
patch(
"daemon.handlers.status.status_pending",
return_value=_pending(
firewall=True,
dnsmasq=True,
nginx=True,
wireguard=True,
networkd=True,
),
),
patch("daemon.handlers.status.refresh_state"),
patch("daemon.handlers.status.revert_to_applied", side_effect=fake_revert),
patch(
"daemon.handlers.status._config_path",
side_effect=lambda name: paths[name],
),
):
result = status.status_cancel_all(None, None)
assert result["cancelled"] == status.SYS_ORDER
assert call_order == status.SYS_ORDER
+437
View File
@@ -0,0 +1,437 @@
"""Tests for daemon/handlers/status.py — aggregate pending + apply-all."""
from typing import Any, ClassVar
from unittest.mock import MagicMock, patch
from daemon.handlers import status
from lib.state import State
def _make_state(**kwargs):
"""Create a minimal in-memory state for pending checks."""
st = State()
for name, data in kwargs.items():
st.set(name, data)
return st
def _mock_state_store(state_dict):
"""Return a mock that looks like state_store.get()."""
mock = MagicMock()
mock.get.side_effect = lambda name: state_dict.get(name)
return mock
class TestFwChangeSummary:
"""Test fw_change_summary helper from status module."""
def test_interfaces_summary(self):
s = status.fw_change_summary(
"internal", "interfaces", {"config": ["eth1"], "live": []}
)
assert "Zone internal: interfaces changed" in s
assert "eth1" in s
def test_services_summary(self):
s = status.fw_change_summary(
"dmz", "services", {"config": ["ssh", "dns"], "live": ["ssh"]}
)
assert "Zone dmz: services changed" in s
def test_rich_rules_summary(self):
s = status.fw_change_summary(
"public", "rich_rules", {"config_count": 2, "live_count": 1}
)
assert "config: 2" in s
assert "live: 1" in s
def test_forward_ports_summary(self):
s = status.fw_change_summary(
"wan", "forward_ports", {"config_count": 3, "live_count": 0}
)
assert "Zone wan: port forwards differ" in s
def test_masquerade_summary(self):
s = status.fw_change_summary(
"lan", "masquerade", {"config": True, "live": False}
)
assert "Zone lan: masquerade changed" in s
def test_target_summary(self):
s = status.fw_change_summary(
"vpn", "target", {"config": "ACCEPT", "live": "default"}
)
assert "Zone vpn: target changed" in s
def test_unknown_type_summary(self):
s = status.fw_change_summary("public", "weird", {})
assert "Zone public: weird changed" in s
class TestHashSubsystem:
"""Test _hash_subsystem helper from status module."""
def test_no_state(self):
result = status._hash_subsystem("nginx", None)
assert result["pending_changes"] is False
assert result["summary"] == "Up to date"
def test_pending_true(self):
st = {"status": {"pending_changes": True}}
result = status._hash_subsystem("wireguard", st)
assert result["pending_changes"] is True
assert "unapplied changes" in result["summary"]
assert len(result["changes"]) == 1
def test_pending_false(self):
st = {"status": {"pending_changes": False}}
result = status._hash_subsystem("networkd", st)
assert result["pending_changes"] is False
def test_empty_status(self):
st = {}
result = status._hash_subsystem("dnsmasq", st)
assert result["pending_changes"] is False
class TestStatusPending:
"""Test the aggregate pending endpoint."""
@patch("daemon.handlers.status.state_store")
def test_all_synced(self, mock_store):
mock_store.get.return_value = {
"firewall": {"pending": {"needs_apply": False, "pending": []}},
"dnsmasq": {"status": {"pending_changes": False}},
"nginx": {"status": {"pending_changes": False}},
"wireguard": {"status": {"pending_changes": False}},
"networkd": {"status": {"pending_changes": False}},
}
result = status.status_pending(None, None)
assert result["total_changes"] == 0
assert not result["firewall"]["needs_apply"]
assert not result["dnsmasq"]["pending_changes"]
def _patch_store(self, data):
mock = MagicMock()
mock.get.side_effect = lambda name: data.get(name)
return patch("daemon.handlers.status.state_store", mock)
def test_firewall_pending_only(self):
with self._patch_store(
{
"firewall": {
"pending": {
"needs_apply": True,
"pending": [
{
"zone": "internal",
"type": "interfaces",
"config": ["eth1"],
"live": [],
}
],
}
},
"dnsmasq": {"status": {"pending_changes": False}},
"nginx": {"status": {"pending_changes": False}},
"wireguard": {"status": {"pending_changes": False}},
"networkd": {"status": {"pending_changes": False}},
}
):
result = status.status_pending(None, None)
assert result["total_changes"] == 1
assert result["firewall"]["change_count"] == 1
def test_multiple_subsystems_pending(self):
with self._patch_store(
{
"firewall": {
"pending": {
"needs_apply": True,
"pending": [
{
"zone": "lan",
"type": "services",
"config": ["ssh"],
"live": [],
},
{
"zone": "wan",
"type": "masquerade",
"config": True,
"live": False,
},
],
}
},
"dnsmasq": {"status": {"pending_changes": True}},
"nginx": {"status": {"pending_changes": False}},
"wireguard": {"status": {"pending_changes": True}},
"networkd": {"status": {"pending_changes": False}},
}
):
result = status.status_pending(None, None)
assert result["total_changes"] == 4 # 2 FW + 1 DHCP + 1 WG
assert result["firewall"]["change_count"] == 2
assert result["firewall"]["needs_apply"] is True
assert result["dnsmasq"]["pending_changes"] is True
assert result["wireguard"]["pending_changes"] is True
def test_empty_state(self):
with self._patch_store({}):
result = status.status_pending(None, None)
assert result["total_changes"] == 0
assert not result["firewall"]["needs_apply"]
def test_firewall_uncovered_advisory_not_counted(self):
with self._patch_store(
{
"firewall": {
"pending": {"needs_apply": False, "pending": []},
"uncovered_interfaces": ["eth1"],
},
"dnsmasq": {"status": {"pending_changes": False}},
"nginx": {"status": {"pending_changes": False}},
"wireguard": {"status": {"pending_changes": False}},
"networkd": {"status": {"pending_changes": False}},
}
):
result = status.status_pending(None, None)
assert result["firewall"]["uncovered_interfaces"] == ["eth1"]
assert result["firewall"]["coverage_warnings"]
assert "eth1" in result["firewall"]["coverage_warnings"][0]
# Advisory: must not flip needs_apply or count as a change.
assert not result["firewall"]["needs_apply"]
assert result["firewall"]["change_count"] == 0
assert result["total_changes"] == 0
def test_firewall_no_uncovered_no_warnings(self):
with self._patch_store(
{
"firewall": {
"pending": {"needs_apply": False, "pending": []},
"uncovered_interfaces": [],
},
"dnsmasq": {"status": {"pending_changes": False}},
"nginx": {"status": {"pending_changes": False}},
"wireguard": {"status": {"pending_changes": False}},
"networkd": {"status": {"pending_changes": False}},
}
):
result = status.status_pending(None, None)
assert result["firewall"]["uncovered_interfaces"] == []
assert result["firewall"]["coverage_warnings"] == []
assert result["total_changes"] == 0
def test_firewall_missing_uncovered_key_defaults_empty(self):
with self._patch_store(
{
"firewall": {
"pending": {"needs_apply": False, "pending": []},
},
"dnsmasq": None,
"nginx": None,
"wireguard": None,
"networkd": None,
}
):
result = status.status_pending(None, None)
assert result["firewall"]["uncovered_interfaces"] == []
assert result["firewall"]["coverage_warnings"] == []
def test_firewall_no_pending_key(self):
with self._patch_store(
{
"firewall": {},
"dnsmasq": {"status": {"pending_changes": False}},
"nginx": None,
"wireguard": None,
"networkd": None,
}
):
result = status.status_pending(None, None)
assert result["total_changes"] == 0
assert not result["firewall"]["needs_apply"]
class TestStatusApplyAll:
"""Test the apply-all endpoint.
Patches SYS_APPLY dict entries directly since they hold function
references at import time.
"""
_fake_pending_all: ClassVar[dict[str, Any]] = {
"firewall": {"needs_apply": False, "change_count": 0, "changes": []},
"dnsmasq": {"pending_changes": False, "summary": "Up to date", "changes": []},
"nginx": {"pending_changes": False, "summary": "Up to date", "changes": []},
"wireguard": {"pending_changes": False, "summary": "Up to date", "changes": []},
"networkd": {"pending_changes": False, "summary": "Up to date", "changes": []},
}
@patch("daemon.handlers.status.status_pending")
@patch("daemon.handlers.status.refresh_state")
def test_nothing_to_apply(self, mock_refresh, mock_pending):
mock_pending.return_value = self._fake_pending_all
result = status.status_apply_all(None, None)
assert result["applied"] == []
assert result["errors"] == {}
mock_refresh.assert_called_once()
def test_applies_pending_subsystems(self):
mock_net = MagicMock()
mock_fw = MagicMock()
pending_data = {**self._fake_pending_all}
pending_data["firewall"]["needs_apply"] = True
pending_data["firewall"]["change_count"] = 1
pending_data["networkd"]["pending_changes"] = True
with (
patch("daemon.handlers.status.status_pending", return_value=pending_data),
patch("daemon.handlers.status.refresh_state"),
patch.dict(
"daemon.handlers.status.SYS_APPLY",
{
"networkd": mock_net,
"firewall": mock_fw,
},
),
):
result = status.status_apply_all(None, None)
assert "networkd" in result["applied"]
assert "firewall" in result["applied"]
mock_net.assert_called_once()
mock_fw.assert_called_once()
def test_error_in_subsystem(self):
mock_fw = MagicMock(side_effect=RuntimeError("firewalld not running"))
pending_data = {**self._fake_pending_all}
pending_data["firewall"]["needs_apply"] = True
pending_data["firewall"]["change_count"] = 1
with (
patch("daemon.handlers.status.status_pending", return_value=pending_data),
patch("daemon.handlers.status.refresh_state"),
patch.dict("daemon.handlers.status.SYS_APPLY", {"firewall": mock_fw}),
):
result = status.status_apply_all(None, None)
assert "firewall" not in result["applied"]
assert "Firewall" in result["errors"]
assert "firewalld not running" in result["errors"]["Firewall"]
def test_order_is_respected(self):
call_order = []
def track(name):
def wrapper(*args):
call_order.append(name)
return wrapper
mock_net = MagicMock(side_effect=track("networkd"))
mock_wg = MagicMock(side_effect=track("wireguard"))
pending_data = {**self._fake_pending_all}
pending_data["wireguard"]["pending_changes"] = True
pending_data["networkd"]["pending_changes"] = True
with (
patch("daemon.handlers.status.status_pending", return_value=pending_data),
patch("daemon.handlers.status.refresh_state"),
patch.dict(
"daemon.handlers.status.SYS_APPLY",
{
"networkd": mock_net,
"wireguard": mock_wg,
},
),
):
status.status_apply_all(None, None)
assert call_order == ["networkd", "wireguard"]
def test_partial_failure_still_applies_others(self):
mock_fw = MagicMock(side_effect=RuntimeError("fail"))
mock_nginx = MagicMock()
pending_data = {**self._fake_pending_all}
pending_data["firewall"]["needs_apply"] = True
pending_data["firewall"]["change_count"] = 1
pending_data["nginx"]["pending_changes"] = True
with (
patch("daemon.handlers.status.status_pending", return_value=pending_data),
patch("daemon.handlers.status.refresh_state"),
patch.dict(
"daemon.handlers.status.SYS_APPLY",
{
"firewall": mock_fw,
"nginx": mock_nginx,
},
),
):
result = status.status_apply_all(None, None)
assert "firewall" not in result["applied"]
assert "nginx" in result["applied"]
assert "Firewall" in result["errors"]
mock_nginx.assert_called_once()
def test_force_body_forwarded_to_firewall_only(self):
mock_fw = MagicMock()
mock_nginx = MagicMock()
pending_data = {**self._fake_pending_all}
pending_data["firewall"]["needs_apply"] = True
pending_data["firewall"]["change_count"] = 1
pending_data["nginx"]["pending_changes"] = True
with (
patch("daemon.handlers.status.status_pending", return_value=pending_data),
patch("daemon.handlers.status.refresh_state"),
patch.dict(
"daemon.handlers.status.SYS_APPLY",
{
"firewall": mock_fw,
"nginx": mock_nginx,
},
),
):
status.status_apply_all(None, {"force": True})
mock_fw.assert_called_once_with(None, {"force": True})
mock_nginx.assert_called_once_with(None, None)
def test_no_body_passed_without_force(self):
mock_fw = MagicMock()
pending_data = {**self._fake_pending_all}
pending_data["firewall"]["needs_apply"] = True
pending_data["firewall"]["change_count"] = 1
with (
patch("daemon.handlers.status.status_pending", return_value=pending_data),
patch("daemon.handlers.status.refresh_state"),
patch.dict("daemon.handlers.status.SYS_APPLY", {"firewall": mock_fw}),
):
status.status_apply_all(None, None)
mock_fw.assert_called_once_with(None, None)
class TestSysOrder:
"""Verify SYS_ORDER and SYS_LABELS constants."""
def test_order_network_first(self):
assert status.SYS_ORDER[0] == "networkd"
def test_all_subsystems_present(self):
expected = {"networkd", "firewall", "wireguard", "dnsmasq", "nginx"}
assert set(status.SYS_ORDER) == expected
def test_labels_match(self):
for name in status.SYS_ORDER:
assert name in status.SYS_LABELS
assert name in status.SYS_APPLY
def test_apply_functions_callable(self):
for name in status.SYS_ORDER:
assert callable(status.SYS_APPLY[name])
+1044
View File
File diff suppressed because it is too large Load Diff
+782
View File
@@ -0,0 +1,782 @@
"""Tests for lib/system_import module."""
import json
from pathlib import Path
from unittest.mock import patch
import pytest
from lib import system_import
from lib.common import _APPLY_HASH_KEY, _LAST_APPLIED_CONFIG_KEY, config_hash, save_json
@pytest.fixture
def temp_project(tmp_path):
"""Patch all module-level path constants to tmp_path subdirs."""
originals = {
"PROJECT_DIR": system_import.PROJECT_DIR,
"DNSMASQ_CONF": system_import.DNSMASQ_CONF,
"WG_CONF": system_import.WG_CONF,
"NETWORKD_DIR": system_import.NETWORKD_DIR,
"NGINX_SITES_DIR": system_import.NGINX_SITES_DIR,
}
system_import.PROJECT_DIR = tmp_path
system_import.DNSMASQ_CONF = tmp_path / "etc" / "dnsmasq.d" / "vacuum-wall.conf"
system_import.WG_CONF = tmp_path / "etc" / "wireguard" / "wg0.conf"
system_import.NETWORKD_DIR = tmp_path / "etc" / "systemd" / "network"
system_import.NGINX_SITES_DIR = tmp_path / "data" / "nginx" / "sites-enabled"
yield tmp_path
system_import.PROJECT_DIR = originals["PROJECT_DIR"]
system_import.DNSMASQ_CONF = originals["DNSMASQ_CONF"]
system_import.WG_CONF = originals["WG_CONF"]
system_import.NETWORKD_DIR = originals["NETWORKD_DIR"]
system_import.NGINX_SITES_DIR = originals["NGINX_SITES_DIR"]
# ──────────────────────────────────────────────────────────────────────
# Dnsmasq
# ──────────────────────────────────────────────────────────────────────
class TestImportDnsmasq:
def _write_conf(self, tmp_path, content: str) -> Path:
p = tmp_path / "etc" / "dnsmasq.d"
p.mkdir(parents=True, exist_ok=True)
(p / "vacuum-wall.conf").write_text(content)
return p / "vacuum-wall.conf"
def _read_json(self, tmp_path) -> dict:
p = tmp_path / "config" / "dnsmasq" / "config.json"
return json.loads(p.read_text()) if p.exists() else {}
def test_no_conf_file(self, temp_project):
assert not system_import.import_dnsmasq()
def test_no_markers(self, temp_project, tmp_path):
self._write_conf(tmp_path, "# some random config\nserver=1.1.1.1\n")
assert not system_import.import_dnsmasq()
def test_empty_managed_block(self, temp_project, tmp_path):
self._write_conf(tmp_path, f"{system_import.DNSTART}\n{system_import.DNEND}")
assert system_import.import_dnsmasq()
cfg = self._read_json(tmp_path)
assert cfg["dns"]["upstreams"] == []
assert cfg["dhcp"]["ranges"] == []
def test_upstreams_only(self, temp_project, tmp_path):
conf = (
f"{system_import.DNSTART}\n"
"server=8.8.8.8\n"
"server=1.1.1.1\n"
f"{system_import.DNEND}"
)
self._write_conf(tmp_path, conf)
assert system_import.import_dnsmasq()
cfg = self._read_json(tmp_path)
assert cfg["dns"]["upstreams"] == ["8.8.8.8", "1.1.1.1"]
def test_full_config(self, temp_project, tmp_path):
conf = (
f"{system_import.DNSTART}\n"
"server=8.8.8.8\n"
"server=1.1.1.1\n"
"domain=lan\n"
"expand-hosts\n"
"dhcp-range=set:eth1,192.168.2.100,192.168.2.200,12h\n"
"dhcp-option=tag:eth1,3,192.168.2.1\n"
"dhcp-option=tag:eth1,6,192.168.2.1\n"
"dhcp-host=aa:bb:cc:dd:ee:ff,192.168.2.50,printer\n"
"addr/nas.lan/192.168.2.10\n"
f"{system_import.DNEND}"
)
self._write_conf(tmp_path, conf)
assert system_import.import_dnsmasq()
cfg = self._read_json(tmp_path)
assert cfg["dns"]["upstreams"] == ["8.8.8.8", "1.1.1.1"]
assert cfg["dns"]["domain"] == "lan"
assert len(cfg["dhcp"]["ranges"]) == 1
rng = cfg["dhcp"]["ranges"][0]
assert rng["interface"] == "eth1"
assert rng["start"] == "192.168.2.100"
assert rng["end"] == "192.168.2.200"
assert rng["lease_time"] == "12h"
assert rng["gateway"] == "192.168.2.1"
assert rng["dns"] == "192.168.2.1"
assert len(cfg["dhcp"]["static_leases"]) == 1
lease = cfg["dhcp"]["static_leases"][0]
assert lease["mac"] == "aa:bb:cc:dd:ee:ff"
assert lease["ip"] == "192.168.2.50"
assert lease["hostname"] == "printer"
assert len(cfg["dns"]["custom_records"]) == 1
assert cfg["dns"]["custom_records"][0] == {
"name": "nas.lan",
"address": "192.168.2.10",
}
def test_no_resolv_resets_upstreams(self, temp_project, tmp_path):
conf = f"{system_import.DNSTART}\nno-resolv\n{system_import.DNEND}"
self._write_conf(tmp_path, conf)
assert system_import.import_dnsmasq()
cfg = self._read_json(tmp_path)
assert cfg["dns"]["upstreams"] == []
def test_idempotent(self, temp_project, tmp_path):
conf = f"{system_import.DNSTART}\nserver=8.8.8.8\n{system_import.DNEND}"
self._write_conf(tmp_path, conf)
assert system_import.import_dnsmasq()
assert not system_import.import_dnsmasq()
def test_parse_error_returns_false(self, temp_project, tmp_path):
# Conf with markers — parses fine, so this tests the exception handler
# by mocking _parse_dnsmasq_block to raise
conf = f"{system_import.DNSTART}\nserver=8.8.8.8\n{system_import.DNEND}"
self._write_conf(tmp_path, conf)
with patch(
"lib.system_import._parse_dnsmasq_block", side_effect=ValueError("bad")
):
assert not system_import.import_dnsmasq()
def test_preserves_apply_meta_on_drift(self, temp_project, tmp_path):
# Existing config differs from the live conf and carries apply
# bookkeeping — the rewrite must keep the baseline so pending
# detection and cancel-all survive daemon restarts.
conf = f"{system_import.DNSTART}\nserver=8.8.8.8\n{system_import.DNEND}"
self._write_conf(tmp_path, conf)
cfg_path = tmp_path / "config" / "dnsmasq"
cfg_path.mkdir(parents=True, exist_ok=True)
baseline = {"dns": {"upstreams": ["1.1.1.1"]}}
save_json(
cfg_path / "config.json",
{
**baseline,
_APPLY_HASH_KEY: "old-hash",
_LAST_APPLIED_CONFIG_KEY: baseline,
},
)
assert system_import.import_dnsmasq()
cfg = self._read_json(tmp_path)
assert cfg[_APPLY_HASH_KEY] == "old-hash"
assert cfg[_LAST_APPLIED_CONFIG_KEY] == baseline
assert cfg["dns"]["upstreams"] == ["8.8.8.8"]
def test_stamps_applied_on_first_import(self, temp_project, tmp_path):
# No config file yet: the imported content is the running state,
# so it must be stamped as applied (no phantom pending changes).
conf = f"{system_import.DNSTART}\nserver=8.8.8.8\n{system_import.DNEND}"
self._write_conf(tmp_path, conf)
assert system_import.import_dnsmasq()
cfg = self._read_json(tmp_path)
assert cfg[_APPLY_HASH_KEY] == config_hash(cfg)
assert cfg[_LAST_APPLIED_CONFIG_KEY]["dns"]["upstreams"] == ["8.8.8.8"]
# ──────────────────────────────────────────────────────────────────────
# WireGuard
# ──────────────────────────────────────────────────────────────────────
class TestImportWireguard:
def _write_conf(self, tmp_path, content: str) -> Path:
p = tmp_path / "etc" / "wireguard"
p.mkdir(parents=True, exist_ok=True)
(p / "wg0.conf").write_text(content)
return p / "wg0.conf"
def _read_json(self, tmp_path) -> dict:
p = tmp_path / "config" / "wireguard" / "config.json"
return json.loads(p.read_text()) if p.exists() else {}
def test_no_conf_file(self, temp_project):
assert not system_import.import_wireguard()
def test_basic_interface(self, temp_project, tmp_path):
conf = (
"[Interface]\n"
" PrivateKey = abc123\n"
" Address = 10.137.0.1/24\n"
" ListenPort = 51820\n"
)
self._write_conf(tmp_path, conf)
assert system_import.import_wireguard()
cfg = self._read_json(tmp_path)
assert cfg["interface"]["private_key"] == "abc123"
assert cfg["interface"]["addresses"] == ["10.137.0.1/24"]
assert cfg["interface"]["listen_port"] == 51820
assert cfg["interface"]["name"] == "wg0"
assert cfg["peers"] == {}
def test_full_with_peers(self, temp_project, tmp_path):
conf = (
"[Interface]\n"
" PrivateKey = srv-priv\n"
" Address = 10.137.0.1/24\n"
" ListenPort = 51820\n"
" PostUp = iptables -I FORWARD -i wg0 -j ACCEPT\n"
" PostDown = iptables -D FORWARD -i wg0 -j ACCEPT\n"
"\n"
"[Peer] # alice\n"
" PublicKey = alice-pub\n"
" Endpoint = 203.0.113.1:51820\n"
" AllowedIPs = 0.0.0.0/0\n"
" PersistentKeepalive = 25\n"
"\n"
"[Peer] # bob\n"
" PublicKey = bob-pub\n"
" AllowedIPs = 10.0.0.0/8,172.16.0.0/12\n"
)
self._write_conf(tmp_path, conf)
assert system_import.import_wireguard()
cfg = self._read_json(tmp_path)
assert cfg["interface"]["post_up"] == "iptables -I FORWARD -i wg0 -j ACCEPT"
assert cfg["interface"]["post_down"] == "iptables -D FORWARD -i wg0 -j ACCEPT"
assert "alice" in cfg["peers"]
assert cfg["peers"]["alice"]["public_key"] == "alice-pub"
assert cfg["peers"]["alice"]["endpoint"] == "203.0.113.1:51820"
assert cfg["peers"]["alice"]["allowed_ips"] == ["0.0.0.0/0"]
assert cfg["peers"]["alice"]["persistent_keepalive"] == 25
assert "bob" in cfg["peers"]
assert cfg["peers"]["bob"]["allowed_ips"] == ["10.0.0.0/8", "172.16.0.0/12"]
def test_peer_without_name_uses_pubkey(self, temp_project, tmp_path):
conf = (
"[Interface]\n"
" PrivateKey = srv-priv\n"
" Address = 10.137.0.1/24\n"
" ListenPort = 51820\n"
"\n"
"[Peer]\n"
" PublicKey = anon-pub\n"
" AllowedIPs = 0.0.0.0/0\n"
)
self._write_conf(tmp_path, conf)
assert system_import.import_wireguard()
cfg = self._read_json(tmp_path)
assert "anon-pub" in cfg["peers"]
def test_idempotent(self, temp_project, tmp_path):
conf = (
"[Interface]\n"
" PrivateKey = abc123\n"
" Address = 10.137.0.1/24\n"
" ListenPort = 51820\n"
)
self._write_conf(tmp_path, conf)
assert system_import.import_wireguard()
assert not system_import.import_wireguard()
def test_preserves_apply_meta_on_drift(self, temp_project, tmp_path):
conf = (
"[Interface]\n"
" PrivateKey = abc123\n"
" Address = 10.137.0.1/24\n"
" ListenPort = 51820\n"
)
self._write_conf(tmp_path, conf)
cfg_path = tmp_path / "config" / "wireguard"
cfg_path.mkdir(parents=True, exist_ok=True)
baseline = {"interface": {"listen_port": 51821}, "peers": {}}
save_json(
cfg_path / "config.json",
{
**baseline,
_APPLY_HASH_KEY: "old-hash",
_LAST_APPLIED_CONFIG_KEY: baseline,
},
)
assert system_import.import_wireguard()
cfg = self._read_json(tmp_path)
assert cfg[_APPLY_HASH_KEY] == "old-hash"
assert cfg[_LAST_APPLIED_CONFIG_KEY] == baseline
assert cfg["interface"]["listen_port"] == 51820
def test_stamps_applied_on_first_import(self, temp_project, tmp_path):
conf = (
"[Interface]\n"
" PrivateKey = abc123\n"
" Address = 10.137.0.1/24\n"
" ListenPort = 51820\n"
)
self._write_conf(tmp_path, conf)
assert system_import.import_wireguard()
cfg = self._read_json(tmp_path)
assert cfg[_APPLY_HASH_KEY] == config_hash(cfg)
assert cfg[_LAST_APPLIED_CONFIG_KEY]["interface"]["private_key"] == "abc123"
# ──────────────────────────────────────────────────────────────────────
# Networkd
# ──────────────────────────────────────────────────────────────────────
class TestImportNetworkd:
def _write_network(self, tmp_path, name: str, content: str) -> Path:
p = tmp_path / "etc" / "systemd" / "network"
p.mkdir(parents=True, exist_ok=True)
file_path = p / f"99-{name}.network"
file_path.write_text(content)
return file_path
def _read_json(self, tmp_path) -> dict:
p = tmp_path / "config" / "network" / "config.json"
return json.loads(p.read_text()) if p.exists() else {}
def test_no_network_dir(self, temp_project):
assert not system_import.import_networkd()
def test_no_files(self, temp_project, tmp_path):
(tmp_path / "etc" / "systemd" / "network").mkdir(parents=True, exist_ok=True)
assert not system_import.import_networkd()
def test_basic_interface(self, temp_project, tmp_path):
conf = (
"[Match]\n"
"Name=eth0\n"
"\n"
"[Network]\n"
"DHCP=no\n"
"Addresses=192.168.1.1/24\n"
"Gateway=192.168.1.254\n"
"DNS=8.8.8.8\n"
"DNS=1.1.1.1\n"
)
self._write_network(tmp_path, "eth0", conf)
assert system_import.import_networkd()
cfg = self._read_json(tmp_path)
eth0 = cfg["interfaces"]["eth0"]
assert eth0["dhcp"] == "no"
assert eth0["gateway"] == "192.168.1.254"
assert eth0["dns"] == ["8.8.8.8", "1.1.1.1"]
def test_multiple_interfaces(self, temp_project, tmp_path):
self._write_network(
tmp_path, "eth0", "[Match]\nName=eth0\n\n[Network]\nDHCP=yes\n"
)
self._write_network(
tmp_path, "eth1", "[Match]\nName=eth1\n\n[Network]\nDHCP=no\n"
)
assert system_import.import_networkd()
cfg = self._read_json(tmp_path)
assert "eth0" in cfg["interfaces"]
assert "eth1" in cfg["interfaces"]
assert cfg["interfaces"]["eth0"]["dhcp"] == "yes"
assert cfg["interfaces"]["eth1"]["dhcp"] == "no"
def test_preserves_existing_interfaces(self, temp_project, tmp_path):
# Pre-existing JSON has eth2 with no .network file
cfg_path = tmp_path / "config" / "network"
cfg_path.mkdir(parents=True, exist_ok=True)
save_json(cfg_path / "config.json", {"interfaces": {"eth2": {"dhcp": "no"}}})
self._write_network(
tmp_path, "eth0", "[Match]\nName=eth0\n\n[Network]\nDHCP=yes\n"
)
assert system_import.import_networkd()
cfg = self._read_json(tmp_path)
assert "eth0" in cfg["interfaces"]
assert "eth2" in cfg["interfaces"]
def test_idempotent(self, temp_project, tmp_path):
self._write_network(
tmp_path, "eth0", "[Match]\nName=eth0\n\n[Network]\nDHCP=yes\n"
)
assert system_import.import_networkd()
assert not system_import.import_networkd()
def test_address_section_parsed(self, temp_project, tmp_path):
conf = (
"[Match]\n"
"Name=eth0\n"
"\n"
"[Network]\n"
"DHCP=no\n"
"\n"
"[Address]\n"
"Address=192.168.1.1/24\n"
)
self._write_network(tmp_path, "eth0", conf)
assert system_import.import_networkd()
cfg = self._read_json(tmp_path)
eth0 = cfg["interfaces"]["eth0"]
assert "192.168.1.1/24" in eth0.get("addresses", [])
def test_route_section_parsed(self, temp_project, tmp_path):
conf = (
"[Match]\n"
"Name=eth0\n"
"\n"
"[Network]\n"
"DHCP=no\n"
"\n"
"[Route]\n"
"Destination=10.0.0.0/8\n"
"Gateway=192.168.1.254\n"
"Metric=100\n"
)
self._write_network(tmp_path, "eth0", conf)
assert system_import.import_networkd()
cfg = self._read_json(tmp_path)
eth0 = cfg["interfaces"]["eth0"]
assert len(eth0.get("routes", [])) == 1
route = eth0["routes"][0]
assert route["destination"] == "10.0.0.0/8"
assert route["gateway"] == "192.168.1.254"
assert route["metric"] == 100
# ──────────────────────────────────────────────────────────────────────
# Nginx
# ──────────────────────────────────────────────────────────────────────
class TestImportNginx:
def _write_site(self, tmp_path, domain: str, content: str) -> Path:
p = tmp_path / "data" / "nginx" / "sites-enabled"
p.mkdir(parents=True, exist_ok=True)
file_path = p / f"{domain}.conf"
file_path.write_text(content)
return file_path
def _read_json(self, tmp_path) -> dict:
p = tmp_path / "config" / "nginx" / "config.json"
return json.loads(p.read_text()) if p.exists() else {}
def test_no_sites_dir(self, temp_project):
assert not system_import.import_nginx()
def test_no_files(self, temp_project, tmp_path):
(tmp_path / "data" / "nginx" / "sites-enabled").mkdir(
parents=True, exist_ok=True
)
assert not system_import.import_nginx()
def test_acme_challenge_skipped(self, temp_project, tmp_path):
(tmp_path / "data" / "nginx" / "sites-enabled").mkdir(
parents=True, exist_ok=True
)
(
tmp_path / "data" / "nginx" / "sites-enabled" / "_acme-challenge.conf"
).write_text("# stuff\n")
assert not system_import.import_nginx()
def test_unrecognized_file_skipped(self, temp_project, tmp_path):
self._write_site(tmp_path, "my-site", "# some random nginx config\nserver {}\n")
assert not system_import.import_nginx()
def test_basic_site(self, temp_project, tmp_path):
conf = (
"# Auto-generated by Vacuum Wall — do not edit manually\n"
"# Domain: example.com\n"
"\n"
"server {\n"
" listen 80;\n"
" listen [::]:80;\n"
" server_name example.com;\n"
" return 301 https://$host$request_uri;\n"
"}\n"
"\n"
"server {\n"
" listen 443 ssl;\n"
" listen [::]:443 ssl;\n"
" server_name example.com;\n"
"\n"
" ssl_certificate /home/wall/vacuum-wall/data/acme/example.com/fullchain.cer;\n"
" ssl_certificate_key /home/wall/vacuum-wall/data/acme/example.com/example.com.key;\n"
"\n"
" # / -> 192.168.2.50:8080\n"
" location / {\n"
" auth_basic off;\n"
" proxy_pass http://192.168.2.50:8080;\n"
" }\n"
"}\n"
)
self._write_site(tmp_path, "example.com", conf)
assert system_import.import_nginx()
cfg = self._read_json(tmp_path)
assert "example.com" in cfg["domains"]
dom = cfg["domains"]["example.com"]
assert dom["force_ssl"] is True
assert dom["cert"] == "acme"
assert "/" in dom["paths"]
assert dom["paths"]["/"]["backend"]["host"] == "192.168.2.50"
assert dom["paths"]["/"]["backend"]["port"] == 8080
def test_websocket_path(self, temp_project, tmp_path):
conf = (
"# Auto-generated by Vacuum Wall — do not edit manually\n"
"# Domain: example.com\n"
"\n"
"server {\n"
" listen 443 ssl;\n"
" server_name example.com;\n"
"\n"
" ssl_certificate /data/certs/example.com.crt;\n"
" ssl_certificate_key /data/certs/example.com.key;\n"
"\n"
" # / -> 127.0.0.1:9090\n"
" location / {\n"
" auth_basic off;\n"
" proxy_pass http://127.0.0.1:9090;\n"
" }\n"
"\n"
" # /ws -> 127.0.0.1:9091 (WebSocket)\n"
" location /ws {\n"
" auth_basic off;\n"
" proxy_pass http://127.0.0.1:9091;\n"
" }\n"
"}\n"
)
self._write_site(tmp_path, "example.com", conf)
assert system_import.import_nginx()
cfg = self._read_json(tmp_path)
dom = cfg["domains"]["example.com"]
assert dom["cert"] == "selfsigned"
assert dom["paths"]["/ws"]["is_websocket"] is True
assert dom["paths"]["/ws"]["backend"]["port"] == 9091
def test_idempotent(self, temp_project, tmp_path):
conf = (
"# Auto-generated by Vacuum Wall — do not edit manually\n"
"server {\n"
" listen 443 ssl;\n"
" server_name example.com;\n"
" ssl_certificate /data/acme/example.com/fullchain.cer;\n"
" ssl_certificate_key /data/acme/example.com/example.com.key;\n"
" # / -> 127.0.0.1:9090\n"
" location / {\n"
" auth_basic off;\n"
" proxy_pass http://127.0.0.1:9090;\n"
" }\n"
"}\n"
)
self._write_site(tmp_path, "example.com", conf)
assert system_import.import_nginx()
assert not system_import.import_nginx()
# ──────────────────────────────────────────────────────────────────────
# Firewall
# ──────────────────────────────────────────────────────────────────────
FIREWALL_ZONES_OUTPUT = (
"public (active)\n"
" target: default\n"
" interfaces: eth0 eth1\n"
" sources: \n"
" services: dhcpv6-cidr dns mdns ssh\n"
" ports: \n"
" protocols: \n"
" forward-ports: \n"
" source-ports: \n"
" icmp-blocks: \n"
" rich rules: \n"
"\n"
"internal (active)\n"
" target: DEFAULT\n"
" interfaces: eth2\n"
" sources: \n"
" services: dhcpv6-cidr dns mdns samba-client ssh\n"
" ports: \n"
" protocols: \n"
" forward-ports: \n"
" source-ports: \n"
" icmp-blocks: \n"
" rich rules: \n"
"\n"
"dmz (active)\n"
" target: DROP\n"
" interfaces: \n"
" sources: \n"
" services: dns\n"
" ports: \n"
" protocols: \n"
" forward-ports: \n"
" source-ports: \n"
" icmp-blocks: \n"
" rich rules: \n"
)
class TestImportFirewall:
def _read_json(self, tmp_path) -> dict:
p = tmp_path / "config" / "firewall" / "config.json"
return json.loads(p.read_text()) if p.exists() else {}
def test_no_config_file_and_firewalld_down(self, temp_project, tmp_path):
with patch(
"lib.system_import.run", side_effect=RuntimeError("firewalld not running")
):
assert not system_import.import_firewall()
def test_existing_config_not_overwritten(self, temp_project, tmp_path):
cfg_path = tmp_path / "config" / "firewall"
cfg_path.mkdir(parents=True, exist_ok=True)
save_json(
cfg_path / "config.json", {"zones": {"public": {"interfaces": ["eth0"]}}}
)
assert not system_import.import_firewall()
def test_import_zones(self, temp_project, tmp_path):
with patch("lib.system_import.run", return_value=FIREWALL_ZONES_OUTPUT):
assert system_import.import_firewall()
cfg = self._read_json(tmp_path)
assert "zones" in cfg
assert "public" in cfg["zones"]
assert "internal" in cfg["zones"]
# Live target normalizes to "default" -> the target key is omitted
# (key-absence is the canonical "unmanaged" notation, WI-2).
assert "target" not in cfg["zones"]["public"]
assert cfg["zones"]["public"]["interfaces"] == ["eth0", "eth1"]
assert cfg["zones"]["public"]["services"] == [
"dhcpv6-cidr",
"dns",
"mdns",
"ssh",
]
assert "target" not in cfg["zones"]["internal"]
assert cfg["zones"]["internal"]["interfaces"] == ["eth2"]
def test_import_keeps_nondefault_target(self, temp_project, tmp_path):
# A zone with interfaces and a non-default live target keeps its
# explicit target key (ACCEPT/DROP/REJECT remain fully managed).
output = (
"trusted (active)\n"
" target: ACCEPT\n"
" interfaces: eth3\n"
" sources: \n"
" services: \n"
" ports: \n"
" protocols: \n"
" forward-ports: \n"
" source-ports: \n"
" icmp-blocks: \n"
" rich rules: \n"
)
with patch("lib.system_import.run", return_value=output):
assert system_import.import_firewall()
cfg = self._read_json(tmp_path)
assert cfg["zones"]["trusted"]["target"] == "ACCEPT"
assert cfg["zones"]["trusted"]["interfaces"] == ["eth3"]
def test_empty_interface_zones_skipped(self, temp_project, tmp_path):
with patch("lib.system_import.run", return_value=FIREWALL_ZONES_OUTPUT):
assert system_import.import_firewall()
cfg = self._read_json(tmp_path)
assert "dmz" not in cfg["zones"]
def test_import_stamps_applied(self, temp_project, tmp_path):
# Fresh import adopts the live firewalld state, which is by
# definition the applied state — the file must carry a baseline.
with patch("lib.system_import.run", return_value=FIREWALL_ZONES_OUTPUT):
assert system_import.import_firewall()
cfg = self._read_json(tmp_path)
assert cfg[_APPLY_HASH_KEY] == config_hash(cfg)
assert cfg[_LAST_APPLIED_CONFIG_KEY]["zones"]["public"]["interfaces"] == [
"eth0",
"eth1",
]
def test_parse_error_returns_false(self, temp_project, tmp_path):
with patch("lib.system_import.run", return_value="garbage with no valid zones"):
assert not system_import.import_firewall()
# ──────────────────────────────────────────────────────────────────────
# import_all
# ──────────────────────────────────────────────────────────────────────
class TestImportAll:
def test_all_missing(self, temp_project, tmp_path):
from unittest.mock import MagicMock, patch
mock_run = MagicMock(side_effect=RuntimeError("command not found"))
with patch.object(system_import, "run", mock_run):
result = system_import.import_all()
assert result == []
def test_returns_updated_subsystems(self, temp_project, tmp_path):
# Create dnsmasq conf
etc = tmp_path / "etc" / "dnsmasq.d"
etc.mkdir(parents=True, exist_ok=True)
conf = f"{system_import.DNSTART}\nserver=8.8.8.8\n{system_import.DNEND}"
(etc / "vacuum-wall.conf").write_text(conf)
# Create wireguard conf
wg_etc = tmp_path / "etc" / "wireguard"
wg_etc.mkdir(parents=True, exist_ok=True)
(wg_etc / "wg0.conf").write_text(
"[Interface]\n PrivateKey = abc\n Address = 10.137.0.1/24\n ListenPort = 51820\n"
)
result = system_import.import_all()
assert "dnsmasq" in result
assert "wireguard" in result
assert "network" not in result
assert "nginx" not in result
def test_parse_error_does_not_crash(self, temp_project, tmp_path):
# Create a dnsmasq conf that will parse fine
etc = tmp_path / "etc" / "dnsmasq.d"
etc.mkdir(parents=True, exist_ok=True)
conf = f"{system_import.DNSTART}\nserver=8.8.8.8\n{system_import.DNEND}"
(etc / "vacuum-wall.conf").write_text(conf)
# Make wireguard import fail
wg_etc = tmp_path / "etc" / "wireguard"
wg_etc.mkdir(parents=True, exist_ok=True)
(wg_etc / "wg0.conf").write_text("[Interface]\n")
# This should not raise, just log warning
result = system_import.import_all()
assert "dnsmasq" in result
# ──────────────────────────────────────────────────────────────────────
# _cfgs_equal
# ──────────────────────────────────────────────────────────────────────
class TestCfgsEqual:
def test_equal(self):
assert system_import._cfgs_equal({"a": 1}, {"a": 1})
def test_not_equal(self):
assert not system_import._cfgs_equal({"a": 1}, {"a": 2})
def test_ignores_applied_hash(self):
a = {"a": 1, "_last_applied_hash": "abc"}
b = {"a": 1, "_last_applied_hash": "xyz"}
assert system_import._cfgs_equal(a, b)
def test_nested(self):
a = {"dhcp": {"ranges": [{"start": "1.2.3.4"}]}}
b = {"dhcp": {"ranges": [{"start": "1.2.3.4"}]}}
assert system_import._cfgs_equal(a, b)
# ──────────────────────────────────────────────────────────────────────
# Template marker consistency
# ──────────────────────────────────────────────────────────────────────
class TestTemplateMarker:
"""Verify nginx templates contain the expected auto-generated marker."""
def test_acme_challenge_has_marker(self):
content = (
Path(__file__).resolve().parent.parent
/ "system"
/ "nginx"
/ "acme-challenge.conf"
).read_text()
assert "# Auto-generated by Vacuum Wall" in content
def test_server_block_has_marker(self):
content = (
Path(__file__).resolve().parent.parent
/ "system"
/ "nginx"
/ "server_block.conf"
).read_text()
assert "# Auto-generated by Vacuum Wall" in content
+313 -6
View File
@@ -23,6 +23,13 @@ class TestDefaultConfig:
assert cfg["interface"]["private_key"] == ""
assert cfg["peers"] == {}
def test_classes_have_phase2_fields(self):
cfg = wireguard.DEFAULT_CONFIG
for ck, cv in cfg["access_classes"].items():
assert "subnet" in cv, f"Class {ck} missing subnet"
assert "listen_port" in cv, f"Class {ck} missing listen_port"
assert "lan_access" in cv, f"Class {ck} missing lan_access"
class TestGetConfig:
def test_returns_default_when_no_file(self, temp_config):
@@ -74,6 +81,92 @@ class TestGenerateKeyPair:
assert mock_run.call_args_list[1].kwargs.get("input") == "private-key"
class TestClassHelpers:
def test_class_interface_name(self):
assert wireguard.get_class_interface_name("full") == "wg-full"
assert wireguard.get_class_interface_name("internet") == "wg-internet"
assert wireguard.get_class_interface_name("custom") == "wg-custom"
def test_class_zone_name(self):
assert wireguard.get_class_zone_name("full") == "vpn-full"
assert wireguard.get_class_zone_name("internet") == "vpn-internet"
def test_class_peers(self):
cfg = {
"peers": {
"alice": {"access_class": "full", "public_key": "pk1"},
"bob": {"access_class": "internet", "public_key": "pk2"},
"carol": {"access_class": None, "public_key": "pk3"},
}
}
full_peers = wireguard._class_peers(cfg, "full")
assert len(full_peers) == 1
assert "alice" in full_peers
int_peers = wireguard._class_peers(cfg, "internet")
assert len(int_peers) == 1
assert "bob" in int_peers
class TestGenerateClassConf:
def test_returns_none_when_no_peers(self, temp_config):
cfg = deepcopy(wireguard.DEFAULT_CONFIG)
cfg["access_classes"]["full"]["private_key"] = "test-key"
result = wireguard.generate_class_conf(cfg, "full")
assert result is None
@patch("lib.wireguard.ENV")
def test_renders_template_per_class(self, mock_env, temp_config):
mock_tmpl = MagicMock()
mock_tmpl.render.return_value = "[Interface]\nPrivateKey = x\n"
mock_env.get_template.return_value = mock_tmpl
cfg = deepcopy(wireguard.DEFAULT_CONFIG)
cfg["access_classes"]["full"]["private_key"] = "test-key"
cfg["peers"]["alice"] = {
"access_class": "full",
"public_key": "pub1",
"private_key": "priv1",
"allowed_ips": ["0.0.0.0/0"],
}
result = wireguard.generate_class_conf(cfg, "full")
assert result is not None
assert mock_tmpl.render.call_count == 1
call_kwargs = mock_tmpl.render.call_args.kwargs
assert call_kwargs["interface"]["name"] == "wg-full"
assert call_kwargs["interface"]["private_key"] == "test-key"
assert "alice" in call_kwargs["peers"]
def test_raises_when_no_private_key(self, temp_config):
cfg = {
"interface": {
"name": "wg0",
"listen_port": 51820,
"private_key": "",
"addresses": ["10.137.0.1/24"],
},
"access_classes": {
"full": {
"name": "Full LAN Access",
"description": "Full access",
"subnet": "10.137.0.0/24",
"listen_port": 51820,
"lan_access": True,
},
},
"peers": {
"alice": {
"access_class": "full",
"public_key": "pub1",
},
},
}
wireguard.save_config(cfg)
cfg = wireguard.get_config()
with pytest.raises(ValueError, match="no private key"):
wireguard.generate_class_conf(cfg, "full")
class TestGetPeers:
def test_empty_peers(self, temp_config):
peers = wireguard.get_peers()
@@ -238,12 +331,226 @@ class TestStatus:
class TestGenerateWgShowParser:
def test_parses_peer_output(self):
output = (
"peer: PUBKEY1\n endpoint: 203.0.113.1:51820\n allowed ips: 10.0.0.0/24\n"
"interface: wg0\n"
" public key: IFACE-PUB\n"
" listening port: 51820\n"
" peer: PUBKEY1\n endpoint: 203.0.113.1:51820\n allowed ips: 10.0.0.0/24\n"
)
result = wireguard._parse_wg_show(output)
assert "PUBKEY1" in result
assert result["PUBKEY1"]["endpoint"] == "203.0.113.1:51820"
result = wireguard.parse_wg_show_output(output)
assert result["up"] is True
assert result["interface"]["public_key"] == "IFACE-PUB"
assert result["interface"]["listen_port"] == 51820
assert len(result["peers"]) == 1
assert result["peers"][0]["public_key"] == "PUBKEY1"
assert result["peers"][0]["endpoint"] == "203.0.113.1:51820"
assert result["peers"][0]["allowed_ips"] == ["10.0.0.0/24"]
def test_empty_output(self):
result = wireguard._parse_wg_show("")
assert result == {}
result = wireguard.parse_wg_show_output("")
assert result["up"] is False
assert result["peers"] == []
def test_parses_fwmark(self):
output = (
"interface: wg0\n"
" public key: IFACE-PUB\n"
" listening port: 51820\n"
" fwmark: 0x0\n"
)
result = wireguard.parse_wg_show_output(output)
assert result["up"] is True
assert result["interface"]["fwmark"] == "0x0"
def test_peer_transfer_and_keepalive(self):
output = (
"interface: wg0\n"
" public key: IFACE-PUB\n"
" listening port: 51820\n"
" peer: PUBKEY1\n"
" endpoint: 203.0.113.1:51820\n"
" allowed ips: 10.0.0.0/24, 10.0.1.0/24\n"
" latest handshake: 2 minutes ago\n"
" transfer: 1.23 GiB received, 4.56 GiB sent\n"
" persistent-keepalive: 25\n"
)
result = wireguard.parse_wg_show_output(output)
peer = result["peers"][0]
assert peer["allowed_ips"] == ["10.0.0.0/24", "10.0.1.0/24"]
assert peer["latest_handshake"] == "2 minutes ago"
assert peer["transfer_received"] == "1.23 GiB received"
assert peer["transfer_sent"] == "4.56 GiB sent"
assert peer["persistent_keepalive"] == 25
def test_bad_keepalive_value(self):
output = "interface: wg0\n peer: PUBKEY1\n persistent-keepalive: bogus\n"
result = wireguard.parse_wg_show_output(output)
assert result["peers"][0]["persistent_keepalive"] is None
class TestAccessClasses:
def test_default_config_has_access_classes(self):
cfg = wireguard.DEFAULT_CONFIG
assert "access_classes" in cfg
assert "full" in cfg["access_classes"]
assert "internet" in cfg["access_classes"]
def test_ensure_access_classes_empty(self, temp_config):
cfg = {"interface": {}, "access_classes": {}, "peers": {}}
wireguard._ensure_access_classes(cfg)
assert "full" in cfg["access_classes"]
assert "internet" in cfg["access_classes"]
def test_ensure_access_classes_preserves_existing(self, temp_config):
cfg = {
"interface": {},
"access_classes": {"custom": {"name": "Custom"}},
"peers": {},
}
wireguard._ensure_access_classes(cfg)
assert "custom" in cfg["access_classes"]
assert "full" not in cfg["access_classes"]
def test_ensure_class_defaults_adds_phase2_fields(self, temp_config):
cfg = {"interface": {}, "access_classes": {"old": {"name": "Old"}}, "peers": {}}
wireguard._ensure_access_classes(cfg)
c = cfg["access_classes"]["old"]
assert "subnet" in c
assert "listen_port" in c
assert "lan_access" in c
assert "private_key" in c
assert "public_key" in c
class TestAddPeerWithNewFields:
@patch("lib.wireguard.generate_keypair")
def test_add_peer_with_description_and_access_class(self, mock_gen, temp_config):
mock_gen.return_value = ("priv", "pub")
result = wireguard.add_peer(
"test-peer",
description="Test peer",
access_class="full",
)
assert result["description"] == "Test peer"
assert result["access_class"] == "full"
cfg = wireguard.get_config()
assert cfg["peers"]["test-peer"]["description"] == "Test peer"
assert cfg["peers"]["test-peer"]["access_class"] == "full"
@patch("lib.wireguard.generate_keypair")
def test_update_peer_preserves_existing_fields(self, mock_gen, temp_config):
mock_gen.return_value = ("priv", "pub")
wireguard.add_peer("p1", description="original", access_class="full")
wireguard.add_peer("p1", endpoint="1.2.3.4:51820")
cfg = wireguard.get_config()
assert cfg["peers"]["p1"]["endpoint"] == "1.2.3.4:51820"
assert cfg["peers"]["p1"]["description"] == "original"
assert cfg["peers"]["p1"]["access_class"] == "full"
class TestInterfaceHasNewFields:
def test_default_has_server_endpoint(self):
assert wireguard.DEFAULT_CONFIG["interface"].get("server_endpoint") == ""
def test_default_has_description(self):
assert wireguard.DEFAULT_CONFIG["interface"].get("description") == ""
class TestClassKeyGeneration:
@patch("lib.wireguard.run_proc")
def test_generates_keypair_for_class(self, mock_run, temp_config):
mock_run.side_effect = [
MagicMock(returncode=0, stdout="class-priv\n"),
MagicMock(returncode=0, stdout="class-pub\n"),
]
# Create config with a fresh class (no prior keys, no auto-merge from file)
cfg = {
"interface": {
"name": "wg0",
"private_key": "existing",
"public_key": "pub",
},
"access_classes": {"fresh": {"name": "Fresh", "description": "New class"}},
"peers": {},
}
wireguard.save_config(cfg)
priv, pub = wireguard.generate_class_keypair("fresh")
assert priv == "class-priv"
assert pub == "class-pub"
loaded = wireguard.get_config()
assert loaded["access_classes"]["fresh"]["private_key"] == "class-priv"
assert loaded["access_classes"]["fresh"]["public_key"] == "class-pub"
@patch("lib.wireguard.run_proc")
def test_returns_existing_keys(self, mock_run, temp_config):
cfg = deepcopy(wireguard.DEFAULT_CONFIG)
cfg["access_classes"]["full"]["private_key"] = "existing-priv"
cfg["access_classes"]["full"]["public_key"] = "existing-pub"
wireguard.save_config(cfg)
priv, pub = wireguard.generate_class_keypair("full")
assert priv == "existing-priv"
assert pub == "existing-pub"
mock_run.assert_not_called()
def test_raises_for_missing_class(self, temp_config):
with pytest.raises(ValueError, match="not found"):
wireguard.generate_class_keypair("nonexistent")
class TestGetPeerStatusMultiInterface:
@patch("lib.wireguard.run_proc")
def test_aggregates_peers_across_classes(self, mock_run, temp_config):
def side_effect(cmd, **kwargs):
iface = cmd[-1]
if iface == "wg-full":
return MagicMock(
returncode=0,
stdout="interface:\n public key: FULL-PUB\n\npeer: PUB1\n",
)
if iface == "wg-internet":
return MagicMock(
returncode=0,
stdout="interface:\n public key: INT-PUB\n\npeer: PUB2\n",
)
# Legacy interface
if iface == "wg0":
return MagicMock(returncode=1, stdout="")
return MagicMock(returncode=1, stdout="")
cfg = deepcopy(wireguard.DEFAULT_CONFIG)
cfg["access_classes"]["full"]["private_key"] = "full-priv"
cfg["access_classes"]["internet"]["private_key"] = "int-priv"
wireguard.save_config(cfg)
mock_run.side_effect = side_effect
peers = wireguard.get_peer_status()
assert len(peers) == 2
assert peers[0]["public_key"] == "PUB1"
assert peers[0]["access_class"] == "full"
assert peers[1]["public_key"] == "PUB2"
assert peers[1]["access_class"] == "internet"
class TestApplyClass:
@patch("lib.wireguard.run")
@patch("lib.wireguard.run_proc")
@patch("lib.wireguard.ENV")
def test_apply_class_writes_and_up(
self, mock_env, mock_proc, mock_run, temp_config
):
mock_tmpl = MagicMock()
mock_tmpl.render.return_value = "[Interface]\nPrivateKey = x\n"
mock_env.get_template.return_value = mock_tmpl
cfg = deepcopy(wireguard.DEFAULT_CONFIG)
cfg["access_classes"]["full"]["private_key"] = "test-key"
cfg["peers"]["alice"] = {
"access_class": "full",
"public_key": "pub1",
"private_key": "priv1",
"allowed_ips": ["0.0.0.0/0"],
}
wireguard.save_config(cfg)
wireguard.apply_class("full")
# Should call wg-quick up
mock_run.assert_any_call(["wg-quick", "up", "wg-full"], sudo=True)
+47
View File
@@ -0,0 +1,47 @@
"""Tests for WebSocket subprotocol token extraction (daemon.server)."""
import pytest
from daemon.server import _extract_ws_token
# Realistic-looking access token (base64url header.payload.signature).
TOKEN = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiJ9.Y3WNO0CTxb4dlcVCPjUnQi"
class TestExtractWsToken:
def test_raw_jwt_subprotocol(self):
"""Bundled client path: the JWT is sent as the subprotocol name itself."""
assert _extract_ws_token([TOKEN]) == (TOKEN, TOKEN)
def test_raw_jwt_among_other_subprotocols(self):
assert _extract_ws_token(["vacuum-wall", TOKEN]) == (TOKEN, TOKEN)
@pytest.mark.parametrize(
"raw",
["none", "vacuum-wall", "a.b", "a.b.c.d"],
)
def test_no_auth_subprotocol(self, raw):
assert _extract_ws_token([raw]) == (None, None)
def test_empty_list(self):
assert _extract_ws_token([]) == (None, None)
def test_bearer_prefix_form(self):
"""Legacy non-browser form: 'Bearer <token>' subprotocol."""
assert _extract_ws_token([f"Bearer {TOKEN}"]) == (TOKEN, f"Bearer {TOKEN}")
def test_bearer_with_non_jwt_token(self):
assert _extract_ws_token(["Bearer abc123"]) == ("abc123", "Bearer abc123")
def test_bearer_without_token_ignored(self):
assert _extract_ws_token(["Bearer ", "Bearer"]) == (None, None)
def test_jwt_like_name_preferred_over_bearer(self):
"""A JWT-shaped subprotocol wins even when listed after a Bearer one."""
result = _extract_ws_token([f"Bearer {TOKEN}", TOKEN])
assert result == (TOKEN, TOKEN)
def test_subprotocol_with_space_rejected(self):
"""A space is not an RFC 6455 token character — 'Bearer <token>' must
go through the Bearer branch, never the JWT-shape branch."""
assert _extract_ws_token(["a b.c.d"]) == (None, None)
+114
View File
@@ -0,0 +1,114 @@
"""Tests for WS delta structure (daemon.server._poll_loop + broadcasts).
After the push-stream migration the poll loop drives per-subsystem deltas:
a structural diff bumps the version and broadcasts {type: versions,
subsystem, data}; a volatile-only diff broadcasts {type: tick, subsystem,
data}. No legacy `updated` dict / `subsystems` array is emitted.
"""
import asyncio
import hashlib
import json
from contextlib import suppress
from unittest.mock import AsyncMock, MagicMock, patch
import daemon.server as server
from lib.state import State
def _zero_offset_subsystem(interval: int) -> str:
"""Find a subsystem name whose md5 offset is 0 so the loop starts at once."""
for i in range(100_000):
name = f"sub{i}"
offset = int(hashlib.md5(name.encode()).hexdigest(), 16) % interval
if offset == 0:
return name
raise AssertionError("could not find zero-offset name")
def _run_one_poll_iteration(poll_result):
"""Run _poll_loop for a single iteration and return the broadcast mocks."""
namespaced = _zero_offset_subsystem(60)
async def drive():
store = MagicMock()
store.poll.return_value = poll_result
store.bump = MagicMock()
store.get.return_value = {"value": 1}
bv = AsyncMock()
bt = AsyncMock()
task = None
with (
patch.object(server, "state_store", store),
patch.object(server, "broadcast_versions", bv),
patch.object(server, "broadcast_tick", bt),
patch.object(server, "blacklist_expired"),
):
task = asyncio.create_task(server._poll_loop(namespaced, 60))
await asyncio.sleep(0.02) # let one full iteration run
task.cancel()
with suppress(asyncio.CancelledError):
await task
return store, bv, bt
store, bv, bt = asyncio.run(drive())
return store, bv, bt
class TestPollLoopDeltas:
def test_structural_change_broadcasts_versions(self):
store, bv, bt = _run_one_poll_iteration((True, False))
store.bump.assert_called_once_with(_zero_offset_subsystem(60))
bv.assert_awaited_once()
bt.assert_not_awaited()
def test_volatile_change_broadcasts_tick(self):
store, bv, bt = _run_one_poll_iteration((False, True))
store.bump.assert_not_called()
bv.assert_not_awaited()
bt.assert_awaited_once()
def test_no_change_no_broadcast(self):
store, bv, bt = _run_one_poll_iteration((False, False))
store.bump.assert_not_called()
bv.assert_not_awaited()
bt.assert_not_awaited()
class TestDeltaMessageShape:
def test_versions_message_carries_subsystem_and_data(self):
"""broadcast_versions emits {type, subsystem, data} — no `updated`."""
store = State()
store.set("firewall", {"zones": {"public": {}}})
ws = AsyncMock()
ws.send_str = AsyncMock()
server._ws_subscribers.add(ws)
try:
with patch.object(server, "state_store", store):
asyncio.run(server.broadcast_versions("firewall"))
ws.send_str.assert_awaited_once()
msg = json.loads(ws.send_str.call_args[0][0])
assert msg["type"] == "versions"
assert msg["subsystem"] == "firewall"
assert msg["data"] == {"zones": {"public": {}}}
assert "updated" not in msg
finally:
server._ws_subscribers.discard(ws)
def test_tick_message_carries_subsystem_and_data(self):
store = State()
store.set("system", {"load": {"load1": 1.0}})
ws = AsyncMock()
ws.send_str = AsyncMock()
server._ws_subscribers.add(ws)
try:
with patch.object(server, "state_store", store):
asyncio.run(server.broadcast_tick("system"))
ws.send_str.assert_awaited_once()
msg = json.loads(ws.send_str.call_args[0][0])
assert msg["type"] == "tick"
assert msg["subsystem"] == "system"
assert msg["data"] == {"load": {"load1": 1.0}}
assert "subsystems" not in msg
finally:
server._ws_subscribers.discard(ws)
+127
View File
@@ -0,0 +1,127 @@
"""Tests for the WS connect snapshot (daemon.server._handle_ws).
After the push-stream migration, a successful WS handshake sends a full
state snapshot ({type: snapshot, data: {subsystem: state|null, ...}})
instead of the retired {type: init, versions: ...} message.
"""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from lib.state import State
@pytest.fixture(autouse=False)
def db_reset():
"""Isolated in-memory DB so a builtin admin exists for token minting.
Mirrors the autouse _db_reset fixture in tests/test_auth.py (the DB
singleton must be reset and pointed at SQLite :memory: before each test).
"""
import os
from lib.db import get_db, reset_db_for_test
reset_db_for_test()
old_backend = os.environ.pop("VACUUM_WALL_DB_BACKEND", None)
old_path = os.environ.pop("VACUUM_WALL_DB_PATH", None)
old_seed = os.environ.pop("VACUUM_WALL_SEED_BUILTIN_ADMIN", None)
os.environ["VACUUM_WALL_DB_BACKEND"] = "sqlite"
os.environ["VACUUM_WALL_DB_PATH"] = ":memory:"
get_db() # triggers builtin-admin seed on the empty :memory: DB
yield
reset_db_for_test()
if old_backend is not None:
os.environ["VACUUM_WALL_DB_BACKEND"] = old_backend
if old_path is not None:
os.environ["VACUUM_WALL_DB_PATH"] = old_path
if old_seed is not None:
os.environ["VACUUM_WALL_SEED_BUILTIN_ADMIN"] = old_seed
@pytest.fixture
def access_token(db_reset):
"""Mint a real access token for the seeded builtin admin."""
from lib.auth import generate_tokens
tokens = generate_tokens("admin", {"firewall": "rw"})
return tokens["access_token"]
class TestWsSnapshot:
def test_snapshot_sent_on_auth_connect(self, access_token):
"""A valid JWT subprotocol yields a full snapshot after auth."""
import asyncio
import daemon.server as server
store = State()
store.set("firewall", {"zones": {"public": {}}})
store.set("system", {"load": {"load1": 0.1}})
# Remaining subsystems stay None (not populated).
ws = AsyncMock()
ws.prepare = AsyncMock()
ws.send_json = AsyncMock()
request = MagicMock()
request.headers = {"Sec-WebSocket-Protocol": access_token}
with (
patch("aiohttp.web.WebSocketResponse", return_value=ws),
patch.object(server, "state_store", store),
):
asyncio.run(server._handle_ws(request))
ws.send_json.assert_awaited_once()
payload = ws.send_json.call_args[0][0]
assert payload["type"] == "snapshot"
data = payload["data"]
# Every subsystem key is present (push-stream: no `init`/`versions` shape).
for name in State.SUBSYSTEMS:
assert name in data
assert data["firewall"] == {"zones": {"public": {}}}
assert data["system"] == {"load": {"load1": 0.1}}
# Unpopulated subsystems are present but None (partial snapshot).
assert data["dnsmasq"] is None
assert data["wireguard"] is None
def test_no_snapshot_without_token(self):
"""Missing token -> 401 JSON, no WS is opened, no snapshot sent."""
import asyncio
import daemon.server as server
ws = AsyncMock()
request = MagicMock()
request.headers = {}
with patch("aiohttp.web.WebSocketResponse") as mock_ctor:
result = asyncio.run(server._handle_ws(request))
assert result.status == 401
mock_ctor.assert_not_called()
ws.send_json.assert_not_awaited()
def test_no_snapshot_on_invalid_token(self, access_token):
"""A token that fails validation -> 401, no snapshot sent."""
import asyncio
import daemon.server as server
ws = AsyncMock()
request = MagicMock()
request.headers = {"Sec-WebSocket-Protocol": access_token}
with (
patch("aiohttp.web.WebSocketResponse") as mock_ctor,
patch("lib.auth.validate_token", return_value=None),
):
result = asyncio.run(server._handle_ws(request))
assert result.status == 401
mock_ctor.assert_not_called()
ws.send_json.assert_not_awaited()
View File
-8326
View File
File diff suppressed because it is too large Load Diff
Vendored Symlink
+1
View File
@@ -0,0 +1 @@
acme-3.1.3.sh

Some files were not shown because too many files have changed in this diff Show More