- move state collectors from lib/state.py to daemon/collectors/ (7
modules, registration side-effect; daemon/server.py imports the
package before the first populate())
- webui/api: new daemon_route() decorator factory in common.py
collapses the try/except daemon-proxy boilerplate in all 8
blueprints (rules/params/body/transform keep responses identical)
- firewall: interface-coverage invariant — config is the source of
truth for zone interfaces (absent key = empty, no hands-off
zones); pure validate_coverage() enforced at save (400) and apply
(409, force: true overrides), top-level `unmanaged` exemption
- lib: get_config() reads are now pure (no dir creation or writes);
new lib/bootstrap.py creates runtime dirs and persists the
one-shot nginx legacy migration at daemon start, after
system_import (lib.nginx.migrate_config_file)
- lib/common: compute_pending() apply-bookkeeping helper
- daemon: emit_and_refresh() handler helper; refresh_state(bump=) so
/status/refresh no longer bumps versions (poll/mutation only)
- acme: move --log last so acme.sh never treats a real arg as the
log-file argument
- docs: AGENTS.md, config.md, state-model.md, api.md updated;
HARDEN.md dropped (plan implemented); apply-confirm force wording
Tests: 917 passed; ruff check + format clean.
acme:
- acme.sh chmods its tree to owner-only (700/600) every run, which
broke the two-user model: a tree left owner-only by one user made
every acme.sh call of the other exit 2
- normalize_acme_home() reopens group access (sudo chmod g+rwX,
files only — setgid dirs trip RestrictSUIDSGID); _run_acme_preflight
is the choke point before every daemon acme.sh call + startup
- acme service now runs as the daemon user; --log persists the raw CA
transcript; SYS_LOG=6 journals manual issue/renew runs
- timer daily-only: two runs/day landed inside ZeroSSL's 24h
validation backoff (Retry-After: 86400) — a permanent renewal lockout
- _collect_acme no longer raises on cert-list failure; reports
status.error (AcmeState.status) so the certs page can surface it
firewall: re-stamp the applied baseline on live zone mutations
(interfaces/services/rich-rules/masquerade/forward-ports) so cancel-all
reverts to post-mutation state, not a stale install-era snapshot;
set_masquerade syncs the declarative config for existing zones;
add_forward_port records toaddr only with toport
status: apply-all accepts {"force": true} (forwarded to the firewall
apply only); ApplyConfirm force checkbox; applyResultToasts() — the
errors map wins over the 200; ActionButton checks errors before the
success toast; dashboard uses ApplyConfirm
system_import: drift re-imports carry the existing apply-meta; first
import stamps the adopted content as applied (it is the running state)
— no phantom pending changes
nginx: get_config only re-saves when migration actually changed the
config (no more owner/mtime churn on every read)
install: repair mis-owned top-level system dirs (tmpfiles
unsafe-path-transition), warn with a full-repair command for deeper
mis-ownership
daemon/server: loop.get_exception_handler() (aiohttp API fix)
tests: 888 pytest + 24 node passing; ruff clean
Post-DHCP-incident hardening per HARDEN.md.
- apply guard: refuse (ConflictError, `force` overrides) when a
network-managed interface would end up in no zone; absent
`interfaces` key = hands-off, explicit `[]` = unassign-all
- surface `uncovered_interfaces` in firewall state (lo/wg* filtered)
+ advisory in /api/status/pending; zones.js banner + interfaces-picker
last-zone confirm
- target drift (Option A): absent or default-normalizing target is
unmanaged: not diffed, never re-set by apply; create_zone runs
--new-zone first and sets non-default targets only; importer omits
the target key for default zones
- FirewallToDhcpSync keeps stale DHCP ranges and flags them instead of
deleting; `dnsmasq` affected only on a real gateway mutation
- real pre-apply recovery snapshot in data/firewall/rules.json
({timestamp, default_zone, zones, config}); drop the empty post-apply
skeleton
- daemon shutdown: bounded grace for in-flight tasks + suppressed
teardown exception noise on SIGTERM
- also carries the firewall service-descriptions feature
(get_service_descriptions + service_descriptions state field + UI)
- tests + docs across firewall/status/state/sync/schema; ruff clean,
867 passing
- lib.common.revert_to_applied(): restore a config file from its
_last_applied_config snapshot (stamped hash); no baseline -> skip with
reason, file untouched
- firewall config_apply now stamps the applied baseline like the other
subsystems; GET /firewall/config and the state collector strip the
internal _last_applied_* keys
- POST /status/cancel-all + /api/status/cancel-all: revert pending
subsystems, {cancelled, skipped, errors}, partial-failure safe
- dashboard: "Cancel All Changes" button with confirm modal
(CancelConfirm, reuses the pending-changes modal rows); the pending
changes card is hidden entirely when nothing is pending
- tests: revert_to_applied, status_cancel_all, firewall stamping/meta
stripping, /api/status/cancel-all route, node tests for CancelConfirm;
firewall _config_apply tests no longer write the real repo config
- docs: api.md, state-model.md, config.md, hoover.md
- lib/common: stamp_applied() now records a _last_applied_config
snapshot alongside the hash; strip_apply_meta() centralizes
bookkeeping-key stripping; deep_diff() reports field-level changes
- state collectors (dnsmasq/nginx/wireguard/networkd) expose
pending_diff so the dashboard can show exactly which fields
changed since the last apply (wireguard diff excludes
private_key paths)
- dashboard pending-changes card renders per-change lines with a
generic fallback when no snapshot is recorded
- firewall: firewalld built-in zones no longer flagged as
unmanaged; public-zone masquerade skipped in pending changes
since apply drives it via nftables propagation
- schema: PendingChange TypedDict; pending_diff on DnsmasqStatus /
WgStatus; tests in test_common.py, test_firewall.py, test_state.py
- daemon: send full snapshot on connect; versions/tick now carry the
full state of one subsystem (subsystem + data); no legacy
updated/subsystems payloads; refresh_state and POST /status/refresh
broadcast per-subsystem versions with data
- client: modelSet() patches models in place; onMessage/topic refresh
retired; 3s initial-load fallback via new POST /api/status/refresh
- schema: lib/schema.py TypedDicts + hoover/schema.js defaults +
docs/state-model.md as single source of truth for state shapes
- system: poll at 1s, volatile metrics registered, dashboard uses a
dedicated system model (status model removed)
- firewall: refuse to strip both https and ssh from the default zone
(409, force override via UI confirm); set_zone_services persists
services to the declarative config; collector exposes default_zone
- UI: pages migrate to flat state shapes; post-mutation modelFetch
refreshes removed (WS delta covers it)
- tests: ws snapshot/delta/broadcast, refresh-state, schema types,
model-set/js ws handler and reconnect fallback
systemd: pre-create volatile /run paths so vacuum-walld's ProtectSystem=strict namespace setup cannot fail with 226/NAMESPACE — RuntimeDirectory=vacuum-wall nginx plus a tmpfiles.d spec (installed to /etc/tmpfiles.d/) covering /run/firewalld and /run/nginx.pid. Drop /run/sudo from ReadWritePaths: NOPASSWD children never need it, and its absence crash-looped restarts after sudo removed /run/sudo.
webui: run the auth session check before mounting the shell so logged-out visitors never flash the sidebar or a protected page; router guard and sidebar now react to auth state, and the login page renders full-bleed.
ws: cap refresh->reconnect episodes at 2 consecutive failures; if the WS path stays dead after a token refresh, abandon reconnection instead of looping refreshAuth forever (UI keeps working via REST until reload).
api: GET /api/network/interfaces now includes loopback and returns per-interface {config, runtime}; dashboard reads runtime.state (carrier counts as up) and the interfaces page filters lo client-side.
daemon: re-collect nginx state after lazy config migration (cached list went stale when the on-disk format changed under it), skip system_import.nginx when config.json already exists (re-parsing vacuum-wall's own generated sites is lossy), and poll nginx (60s) / acme (300s) state so file drift self-heals.
Auth seeding (last-resort guard)
- `_seed_builtin_admin()` in get_db() now skips when
VACUUM_WALL_SEED_BUILTIN_ADMIN=0 or when the users table already
contains any user — previously a fresh service start after a non-default
bootstrap (e.g. --mgmt-user alice) seeded a hard-coded `admin` with an
unrecoverable random password, shadowing the operator's account
- bootstrap_auth.py sets VACUUM_WALL_SEED_BUILTIN_ADMIN=0: bootstrap
creates the operator user itself on a fresh install, so exactly one
account exists and no seeded admin can appear
Frontend (session recovery)
- on page load/restore the in-memory TTL timer is gone, so a valid
7-day refresh token could sit in sessionStorage while the access token
is already expired server-side: the session `check` now attempts
exactly one refresh (POST /api/auth/refresh with the stored refresh
token) on 401 before treating the session as dead
- extract shared `_doRefresh()` used by both the `check` 401 fallback and
the `refresh` action (removes the duplicated rotation logic)
Tests
- update seeding tests to the new any-user-present check; add
test_seed_skipped_when_users_exist, test_seed_skipped_via_env,
test_bootstrap_flow_creates_exactly_one_user, and the auth-model JS
test suite (tests/test-auth-model.js)
Docs
- AGENTS.md: document VACUUM_WALL_SEED_BUILTIN_ADMIN
- architecture.md / hoover.md / security.md: describe the bootstrap
check 401 → one-refresh fallback path
Refresh/logout and token robustness
- drop the post-rotation refresh_tokens row delete in auth_refresh so
logout blacklists the current (rotated) refresh token; remove the
dead _clear_refresh_token_after_rotation helper and clear_active_refresh_token
- reject non-object JWT payloads in _extract_unverified_sub so crafted
Authorization headers return 401 instead of crashing with 500
SQLite user store
- make builtin-admin seeding idempotent: on a concurrent first start the
losing seeder re-checks, finds the winner, and returns instead of
raising IntegrityError
- per-thread sqlite connections + busy_timeout so Flask worker threads
don't hit cross-thread ProgrammingError / SQLITE_BUSY
- add LogsDirectory + /var/log/vacuum-wall to ReadWritePaths in both
systemd units so the fallback admin password actually lands on disk
Frontend
- skip apiFetch 401-recovery for public auth endpoints so a failed
login no longer logs out a valid session
- add /passkeys to the nav (passkey registration was unreachable);
remove the dead checkWebAuthnCapable export
- drop the CSP-blocked inline WS-URL script and the
__WS_URL_PLACEHOLDER__ plumbing; the WS URL is derived from location
Daemon / WS
- parse Sec-WebSocket-Protocol manually (web.Request.get_subprotocols
does not exist in aiohttp 3.13); X-Auth-Token is a custom-nginx
fallback only — docstring and security docs corrected
Install / system
- bootstrap_auth.py is now idempotent: preserves existing auth config
and syncs the admin password on re-runs (new reset_password helper)
- WebUI server block renders auth_basic off (the UI is JWT-protected)
- install.sh chown/chmod skips .git to avoid git dubious-ownership
breakage
- tolerate unreadable /etc/wireguard during system import
Contracts / docs
- create_user returns 409 on duplicate username per docs/api.md
- correct docs/api.md response shapes, docs/security.md blacklist
cleanup wording + one-refresh-per-user caveat, stale WS-URL
references, and the .htpasswd description
Tests: +7 regression tests (rotation/logout revocation, crafted-token
401, concurrent seeding); placeholder-substitution tests replaced with
serve-as-is SPA root tests.
Add require_session parameter to validate_token to enforce session_id
matching for refresh operations. Attacker with stolen refresh token can
no longer bypass session binding by omitting session_id from request.
Also adds backend guard against deleting builtin admin user (was only
blocked at Flask blueprint layer), and removes unused _ALL_RW variable.
The warning message referenced 'data/auth.log' but the actual file is
/var/log/vacuum-wall/auth.log. Also add error logging when the write
fails, instead of silently swallowing the OSError.
- update_permissions: swap to upsert-first-then-delete-stale so a
failed upsert mid-loop rolls back cleanly, leaving the user's
permissions intact. Adds Q_DELETE_PERMISSION_SUBSYSTEM for
targeted removal.
- auth_refresh: generate and persist the new refresh token before
blacklisting/clearing the old one, so a failure in generate_tokens
doesn't leave the user locked out with no valid refresh token.
- Add session_id to refresh tokens and enforce it during validation,
preventing stolen refresh tokens from being usable without the
originating browser session
- Set router.isAuthenticated via auth:login event after successful
login (previously only set at page load)
- Add console.warn logging to WS message parse/handler errors
- Improve _refreshPromise error handling in token refresh flow
- Document rate limiter in-memory limitation and CSP connect-src
same-origin requirement
- Add 3 tests for session-bound refresh token validation
- Truncate admin password in logs; write full password to data/auth.log (0o600)
- Persist access token in sessionStorage so it survives page reloads
- Simplify tryRefreshToken to use GSAP-style promise deduplication
- Remove spurious POST redirect on 401 during token refresh
- Guard passkey button reference in login finally block
- Make session_id parameter optional in validate_token — only enforced when
provided, allowing WebSocket auth which cannot carry custom headers
- Remove decode_token round-trip from daemon WS handler
- Override WebAuthn registration username from JWT user_ctx to prevent users
from registering credentials under another user's account
- Frontend no longer sends username for WebAuthn registration
- Redirect to login on 401 after token refresh fails for POST requests
- Remove redundant auth session check from login page load
- Add tests for session_id semantics and WebAuthn ownership guard
Browsers cannot send custom X-Session-Id header on WebSocket connections,
so decode the token payload to extract session_id. Add WebAuthn
success/failure recording to support rate limiter counter resets.
- Raise ValueError on missing request body in auth_logout
- Add username check in verify_authentication to prevent credential reuse
- Replace N+1 queries in list_users with single JOIN query
Session binding was bypassable: if the X-Session-Id header was absent,
validate_token skipped the check entirely, allowing a stolen JWT to be
used without the originating session.
Server-side: reject 401 early in Flask middleware and daemon WebSocket
handler when X-Session-Id is missing, before calling validate_token.
Updated validate_token to always enforce session_id matching for access
tokens (refresh tokens are unaffected as they carry no session_id claim).
Frontend: removed dead if (stored.session_id) guards in api.js since
the header is now always required. Added X-Session-Id to logout request
headers and always store session_id on login/refresh.
The blacklist table grew indefinitely since cleanup only ran on
password changes and user deletions. Now each blacklist_token()
call has a 2% chance of triggering blacklist_expired() to prune
expired entries. Redundant cleanup calls in update_password()
and delete_user() are removed.
- Rate limiter tracks failures only; success resets counter
- Record failures/successes after password verification, not before
- WebAuthn rp_id/origin resolved dynamically from request domain
- Management domains auto-discovered from nginx backend config
- All WebAuthn operations validate domain against management list
- Add GET /api/auth/webauthn/capable endpoint for frontend checks
- Frontend checkWebAuthnCapable() function for domain-gated UI
- Timing side-channel fix: pre-compute dummy hash at module load
- Builtin admin seeded with random password (logged at WARNING)
- Logout handler returns consistent response shape
- update_permissions now calls blacklist_active_refresh_token and
rotate_user_secret to immediately invalidate stale tokens
- create_user uses returned id from tx.run_one instead of redundant SELECT
- websocket reconnect explicitly closes old connection after token refresh
to prevent onclose handler race condition
- Pass client IP (X-Real-IP header) through Flask to daemon for both
password login and WebAuthn authenticate-finish endpoints
- Rate limiter now checks both IP and username buckets: IP layer catches
enumeration/brute-force attacks across multiple usernames; username
layer protects against single-account targeting from multiple IPs
- Add _wsRefreshing flag to prevent double-scheduling reconnect when
onclose fires during token refresh; simplify async IIFE to .then()/.catch()
- Reset _wsRefreshing on websocket onopen for safety
webui/static/hoover/api.js
Guard retryRes.json() with .catch(() => null) so non-JSON
responses (e.g. nginx 502/503) don't throw and lose the
actual status code. Falls back to 'HTTP <status>' error string.
lib/db_sqlite.py
Replace unsafe sql.split(';') loop with conn.executescript()
which properly handles semicolons inside string literals.
webui/server.py
Add _AUTH_PERSONAL set and _is_personal_auth() so personal
auth operations (session, password, logout, webauthn creds)
skip subsystem permission checks. Users with only firewall:read
can now manage their own credentials without needing auth:rw.
Apply ruff line-wrapping formatting to docs and test files.
Clarify auth middleware: extract user_permissions once before
subsystem check, removing conditional variable scoping.
- WireGuard: refactor to multi-interface 'access classes' model; extract config
generation and helpers into lib/wireguard.py; add per-class up/down endpoints
and API routes; update UI with class management pages and QR code component
- Firewall: fix zone creation with --new-zone before --set-target; skip
masquerade on public zone; add masquerade propagation for nftables backend
so NAT works when internal zones exit via public
- Network: rename sync event subsystem 'network' -> 'networkd'; always stamp
config hash even when deployment fails (fixes pending-changes detection)
- DHCP: add new API endpoint and update frontend page
- State/Sync: update state collectors and sync buses for new subsystems
- Docs: update API and config documentation for new endpoints and schemas
Add docstrings to all handler functions in daemon/handlers/firewall.py, covering
params, return values, and raised exceptions. Add inline comments to
_config_apply() reconciliation steps and the request body merge order.
Add docstrings across lib/ modules for emit helpers (_emit_str, _emit_int, etc.),
volatile stripping logic, two-layer diff strategy, sync event dispatch, and all
cross-subsystem sync subscribers (DnsToFirewall, WgToFirewall, FirewallToDhcp,
NetworkToAllSync).
Document WireGuard/networkd config parsers and key-value mappers in
system_import.py. Add docstrings to _ep(), Registry.decorator,
setup_logging, and _replace helper across daemon/ and lib/.
Add get_interface_ip() helper to resolve an interface's IPv4 address
via 'ip -o addr show'. Use it to back-propagate gateway into DHCP
ranges so clients receive their default route.
- set_dhcp_range() resolves gateway: explicit > existing range > iface IP
- DnsToFirewallSync and FirewallToDhcpSync sync ensure gateways are set
- Remove automatic masquerade toggle from DnsToFirewallSync
- Fix dnsmasq lease file path to /var/lib/misc/dnsmasq.leases
- Rename lease state field expires_at -> expires (ISO string)
- Add 'ip -o addr show' to sudo whitelist
- lib/system_import: new module to import system configs into JSON at daemon startup
- daemon/server.py: call import_all() during startup for config reconciliation
- daemon/handlers/nginx.py: simplify add_domain auth handling, remove duplicate code
- scripts/install.sh: replace inline Python setup with curl-based daemon API calls; apply IP forwarding at runtime
- hoover: bump internal asset versions to v=8
- pages: bump asset versions to v=9
- update-vendor.sh now creates webui/vendor symlinks (htm.js)
- install.sh calls update-vendor.sh after package install
- Add vendor/.empty and webui/vendor/.empty as directory placeholders in git
Add EventBus with loop guards to keep firewall, dnsmasq, wireguard,
and network configs consistent. Handlers emit SyncEvent after mutations;
subscribers compute diffs and write JSON without manual cascade loops.
- Replace per-zone --list-all calls with single --list-all-zones in _collect_firewall
- Add _parse_all_zones_output() parser with rich rules/rich-rules normalization
- Convert daemon shutdown to async with proper runner cleanup and socket unlink
- Add TimeoutStopSec=15 to vacuum-walld.service for graceful stop
- Fix exception handling in _collect_dnsmasq
- Remove management badge from proxy path rows
Replace the legacy top-level management key with a unified paths-based
model. Each domain now contains a paths map where each entry defines its
own backend, auth, headers, and flags (is_management, is_websocket).
- Add _migrate_config() to auto-migrate legacy formats on first load
- Remove set_management_proxy() and POST_NGINX_MANAGEMENT endpoint
- Update server_block.conf template to iterate paths with per-location auth
- Update daemon handler, API blueprint, state collector, and install script
- Add server config generation tests for paths, WebSocket, auth inheritance
- Update frontend proxy page to display per-path rows with flags
- Add find_cert_dir() to resolve both RSA and ECC (domain_ecc/) cert dirs
- Copy acme deploy hook to /deploy/ where acme.sh resolves it
- _parse_account_conf checks both legacy .account.conf and declarative config
- Skip public DNS check when all local IPs are private (NAT)
- Improve check message strings for validity and expiry status
- Support timezone-aware date formats in _days_until parsing
- Filter out "no" SAN domains in cert listing
- Bump frontend asset version cache keys
- Fix DOMContentLoaded race condition in app.js boot
- Fix spread operator in certs.js modal template
- Move acme.sh utilities (_run_acme, _find_acme, etc.) from lib/state to lib/acme
- Rewrite _parse_list_output to support pipe, tab, and column-separated formats
- Add ConflictError (409) to block issuing when cert already exists
- Move _find_issuance helper to detect in-progress issuance per domain
- Update issue_cert to check existing certs and return issuance status
- Fix start_polling to accept event loop explicitly
- Add sudoers entry for chown on vacuum-wall.conf
- Extend systemd ReadWritePaths for /run/nginx.pid and /var/log/nginx
- Update frontend to handle 'existing' issuance status
- lib/state.py: per-subsystem collectors with versioned state store
- daemon/server.py: state refresh on request, batch routing updates
- webui/static/hoover/html.js: new html tag template helper via htm.js
- webui/static/hoover/websocket.js: real-time state change notifications
- webui/static/hoover/vdom.js: VDOM improvements for keyed diff
- All frontend pages refactored to use html templates
- Add tests for state management and polling
- Update docs and AGENTS.md
- Register, view, and deactivate ACME accounts via API and UI
- 16-check validation framework for certificate issuance readiness
- DNS resolution, port, nginx, and firewall pre-flight checks
- External IP detection with NAT support and fallback providers
- Account card and settings modal in certificates page
- Guard certificate issuance behind account registration
- Update modal CSS to overlay-based approach
- 1000+ lines of tests for validation and account handlers
Phase 1-4: Networkd subsystem
- lib/network.py: systemd-networkd config renderer (.network INI files)
with full schema support: [Match], [Link], [Network], [Address], [Route],
[DHCPv4], [DHCPv6] sections. One Address/=DNS= line per value per spec.
Route sections use #N suffix per systemd.syntax(7).
- lib/network.py: generate_network_files() with 50-<name>.network prefix
and stale file cleanup
- lib/network.py: collect_upstream_dns() filters local/private DNS
- lib/network.py: infer_dhcp_ranges() and infer_zones() helpers
- daemon/handlers/network.py: routes for GET/POST /network/interfaces
and full apply with DNS upstream sync to dnsmasq
- webui/api/network.py: Flask blueprint for /api/network/* endpoints
- webui/api: interfaces page updated with IP config inline editing
- lib/state.py: networkd collector using parse_networkctl_status()
- system/sudoers.d/vacuum-walld: networkctl + systemd-network rules
- system/systemd/vacuum-walld.service: ReadWritePaths for /etc/systemd/network
- install.sh: ACME email now optional, configured from WebUI
- lib/acme.py: get_email() falls back to declarative config
Phase 5: Code review fixes
- daemon/server.py: path params now win over JSON body and query params
in request body merge (prevents config save name override)
- daemon/server.py: remove dead 'import re'
- daemon/handlers/network.py: replace Path.mkdir() with sudo mkdir
for /etc/systemd/network (ProtectSystem=strict compatibility)
- system/sudoers.d/vacuum-walld: pin systemctl to specific commands
(reload/is-active dnsmasq instead of wildcard)
- system/sudoers.d/vacuum-walld: restore !requiretty and section comment
- lib/network.py: remove unused _MANAGEMENT_PORTS constant
- webui/api/network.py: remove redundant body[\name\] = name in save_interface
Tests: 332 passing (110 new/updated), ruff clean
Add comprehensive docstrings to firewall, DHCP, proxy, wireguard, certs,
and logs API endpoints. Document parameters, return values, and error cases
for the documentation system.
mteehan