faa076370dd113637da9f780df6ed16722d87c51
26 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 mteehan
|
faa076370d |
refactor: daemon collectors, thin webui proxies, pure config reads
- move state collectors from lib/state.py to daemon/collectors/ (7 modules, registration side-effect; daemon/server.py imports the package before the first populate()) - webui/api: new daemon_route() decorator factory in common.py collapses the try/except daemon-proxy boilerplate in all 8 blueprints (rules/params/body/transform keep responses identical) - firewall: interface-coverage invariant — config is the source of truth for zone interfaces (absent key = empty, no hands-off zones); pure validate_coverage() enforced at save (400) and apply (409, force: true overrides), top-level `unmanaged` exemption - lib: get_config() reads are now pure (no dir creation or writes); new lib/bootstrap.py creates runtime dirs and persists the one-shot nginx legacy migration at daemon start, after system_import (lib.nginx.migrate_config_file) - lib/common: compute_pending() apply-bookkeeping helper - daemon: emit_and_refresh() handler helper; refresh_state(bump=) so /status/refresh no longer bumps versions (poll/mutation only) - acme: move --log last so acme.sh never treats a real arg as the log-file argument - docs: AGENTS.md, config.md, state-model.md, api.md updated; HARDEN.md dropped (plan implemented); apply-confirm force wording Tests: 917 passed; ruff check + format clean. |
||
|
mteehan
|
75b86fd60d |
fix: ACME ownership self-heal + daily timer, apply-all force, firewall baseline re-stamp
acme:
- acme.sh chmods its tree to owner-only (700/600) every run, which
broke the two-user model: a tree left owner-only by one user made
every acme.sh call of the other exit 2
- normalize_acme_home() reopens group access (sudo chmod g+rwX,
files only — setgid dirs trip RestrictSUIDSGID); _run_acme_preflight
is the choke point before every daemon acme.sh call + startup
- acme service now runs as the daemon user; --log persists the raw CA
transcript; SYS_LOG=6 journals manual issue/renew runs
- timer daily-only: two runs/day landed inside ZeroSSL's 24h
validation backoff (Retry-After: 86400) — a permanent renewal lockout
- _collect_acme no longer raises on cert-list failure; reports
status.error (AcmeState.status) so the certs page can surface it
firewall: re-stamp the applied baseline on live zone mutations
(interfaces/services/rich-rules/masquerade/forward-ports) so cancel-all
reverts to post-mutation state, not a stale install-era snapshot;
set_masquerade syncs the declarative config for existing zones;
add_forward_port records toaddr only with toport
status: apply-all accepts {"force": true} (forwarded to the firewall
apply only); ApplyConfirm force checkbox; applyResultToasts() — the
errors map wins over the 200; ActionButton checks errors before the
success toast; dashboard uses ApplyConfirm
system_import: drift re-imports carry the existing apply-meta; first
import stamps the adopted content as applied (it is the running state)
— no phantom pending changes
nginx: get_config only re-saves when migration actually changed the
config (no more owner/mtime churn on every read)
install: repair mis-owned top-level system dirs (tmpfiles
unsafe-path-transition), warn with a full-repair command for deeper
mis-ownership
daemon/server: loop.get_exception_handler() (aiohttp API fix)
tests: 888 pytest + 24 node passing; ruff clean
|
||
|
mteehan
|
ac52918df5 |
firewall: interface-coverage apply guard, target drift, non-destructive DHCP sync
Post-DHCP-incident hardening per HARDEN.md.
- apply guard: refuse (ConflictError, `force` overrides) when a
network-managed interface would end up in no zone; absent
`interfaces` key = hands-off, explicit `[]` = unassign-all
- surface `uncovered_interfaces` in firewall state (lo/wg* filtered)
+ advisory in /api/status/pending; zones.js banner + interfaces-picker
last-zone confirm
- target drift (Option A): absent or default-normalizing target is
unmanaged: not diffed, never re-set by apply; create_zone runs
--new-zone first and sets non-default targets only; importer omits
the target key for default zones
- FirewallToDhcpSync keeps stale DHCP ranges and flags them instead of
deleting; `dnsmasq` affected only on a real gateway mutation
- real pre-apply recovery snapshot in data/firewall/rules.json
({timestamp, default_zone, zones, config}); drop the empty post-apply
skeleton
- daemon shutdown: bounded grace for in-flight tasks + suppressed
teardown exception noise on SIGTERM
- also carries the firewall service-descriptions feature
(get_service_descriptions + service_descriptions state field + UI)
- tests + docs across firewall/status/state/sync/schema; ruff clean,
867 passing
|
||
|
mteehan
|
55309cfd86 |
status: cancel-all reverts pending changes to last applied config
- lib.common.revert_to_applied(): restore a config file from its
_last_applied_config snapshot (stamped hash); no baseline -> skip with
reason, file untouched
- firewall config_apply now stamps the applied baseline like the other
subsystems; GET /firewall/config and the state collector strip the
internal _last_applied_* keys
- POST /status/cancel-all + /api/status/cancel-all: revert pending
subsystems, {cancelled, skipped, errors}, partial-failure safe
- dashboard: "Cancel All Changes" button with confirm modal
(CancelConfirm, reuses the pending-changes modal rows); the pending
changes card is hidden entirely when nothing is pending
- tests: revert_to_applied, status_cancel_all, firewall stamping/meta
stripping, /api/status/cancel-all route, node tests for CancelConfirm;
firewall _config_apply tests no longer write the real repo config
- docs: api.md, state-model.md, config.md, hoover.md
|
||
|
mteehan
|
30b51ad7d3 |
state: applied-config snapshots + per-field pending diffs
- lib/common: stamp_applied() now records a _last_applied_config snapshot alongside the hash; strip_apply_meta() centralizes bookkeeping-key stripping; deep_diff() reports field-level changes - state collectors (dnsmasq/nginx/wireguard/networkd) expose pending_diff so the dashboard can show exactly which fields changed since the last apply (wireguard diff excludes private_key paths) - dashboard pending-changes card renders per-change lines with a generic fallback when no snapshot is recorded - firewall: firewalld built-in zones no longer flagged as unmanaged; public-zone masquerade skipped in pending changes since apply drives it via nftables propagation - schema: PendingChange TypedDict; pending_diff on DnsmasqStatus / WgStatus; tests in test_common.py, test_firewall.py, test_state.py |
||
|
mteehan
|
332d14e37d |
ws: migrate push stream to data streaming
- daemon: send full snapshot on connect; versions/tick now carry the full state of one subsystem (subsystem + data); no legacy updated/subsystems payloads; refresh_state and POST /status/refresh broadcast per-subsystem versions with data - client: modelSet() patches models in place; onMessage/topic refresh retired; 3s initial-load fallback via new POST /api/status/refresh - schema: lib/schema.py TypedDicts + hoover/schema.js defaults + docs/state-model.md as single source of truth for state shapes - system: poll at 1s, volatile metrics registered, dashboard uses a dedicated system model (status model removed) - firewall: refuse to strip both https and ssh from the default zone (409, force override via UI confirm); set_zone_services persists services to the declarative config; collector exposes default_zone - UI: pages migrate to flat state shapes; post-mutation modelFetch refreshes removed (WS delta covers it) - tests: ws snapshot/delta/broadcast, refresh-state, schema types, model-set/js ws handler and reconnect fallback |
||
|
mteehan
|
9c9f92ad04 |
fix: daemon /run spawn hardening, auth guard before first paint, WS refresh cap, interfaces runtime state
systemd: pre-create volatile /run paths so vacuum-walld's ProtectSystem=strict namespace setup cannot fail with 226/NAMESPACE — RuntimeDirectory=vacuum-wall nginx plus a tmpfiles.d spec (installed to /etc/tmpfiles.d/) covering /run/firewalld and /run/nginx.pid. Drop /run/sudo from ReadWritePaths: NOPASSWD children never need it, and its absence crash-looped restarts after sudo removed /run/sudo.
webui: run the auth session check before mounting the shell so logged-out visitors never flash the sidebar or a protected page; router guard and sidebar now react to auth state, and the login page renders full-bleed.
ws: cap refresh->reconnect episodes at 2 consecutive failures; if the WS path stays dead after a token refresh, abandon reconnection instead of looping refreshAuth forever (UI keeps working via REST until reload).
api: GET /api/network/interfaces now includes loopback and returns per-interface {config, runtime}; dashboard reads runtime.state (carrier counts as up) and the interfaces page filters lo client-side.
daemon: re-collect nginx state after lazy config migration (cached list went stale when the on-disk format changed under it), skip system_import.nginx when config.json already exists (re-parsing vacuum-wall's own generated sites is lossy), and poll nginx (60s) / acme (300s) state so file drift self-heals.
|
||
|
mteehan
|
04417cf05c |
WireGuard access classes, firewall nftables fixes, network sync event refactor
- WireGuard: refactor to multi-interface 'access classes' model; extract config generation and helpers into lib/wireguard.py; add per-class up/down endpoints and API routes; update UI with class management pages and QR code component - Firewall: fix zone creation with --new-zone before --set-target; skip masquerade on public zone; add masquerade propagation for nftables backend so NAT works when internal zones exit via public - Network: rename sync event subsystem 'network' -> 'networkd'; always stamp config hash even when deployment fails (fixes pending-changes detection) - DHCP: add new API endpoint and update frontend page - State/Sync: update state collectors and sync buses for new subsystems - Docs: update API and config documentation for new endpoints and schemas |
||
|
mteehan
|
dadabd7954 |
feat: add system metrics dashboard with resource monitoring
- Add system metrics endpoint (CPU load, memory, swap, network traffic) - Collect metrics from /proc and /sys (no subprocess required) - Overhaul dashboard to pull from per-subsystem models - Remove deprecated /status/all monolithic endpoint - Improve networkd import to handle optional priority prefix - Fix CSS duplicate .grid-4 rule and unused dashboard imports |
||
|
mteehan
|
c21639b7f1 |
docs: add comprehensive docstrings and inline comments
Add docstrings to all handler functions in daemon/handlers/firewall.py, covering params, return values, and raised exceptions. Add inline comments to _config_apply() reconciliation steps and the request body merge order. Add docstrings across lib/ modules for emit helpers (_emit_str, _emit_int, etc.), volatile stripping logic, two-layer diff strategy, sync event dispatch, and all cross-subsystem sync subscribers (DnsToFirewall, WgToFirewall, FirewallToDhcp, NetworkToAllSync). Document WireGuard/networkd config parsers and key-value mappers in system_import.py. Add docstrings to _ep(), Registry.decorator, setup_logging, and _replace helper across daemon/ and lib/. |
||
|
mteehan
|
05524f3756 |
fix: critical bugs + security hardening
Phase 1 (critical bugs): - Fix firewall import string-to-list bug (system_import.py) - Add rich rules removal in firewall config apply (handlers/firewall.py) Phase 2 (security hardening): - Restrict sudo wildcards to specific paths (sudoers.d/vacuum-walld) - Fix TOCTOU: use /run/vacuum-wall/ for temp files (nginx, dnsmasq, network handlers) - Remove unnecessary sudo from wg genkey/pubkey (handlers/wireguard.py) Phase 3 (validation): - Validate poll intervals > 0 (daemon/server.py) - Restrict sysctl to whitelisted parameters (handlers/network.py) Phase 4 (defensive programming): - Enforce shell=False in run() and run_proc() (lib/common.py) - Track issuance tasks for graceful shutdown (handlers/acme.py) - Add nginx template marker consistency tests (tests/test_system_import.py) |
||
|
mteehan
|
803258cf18 |
dhcp: auto-populate gateway from interface IP for DHCP ranges
Add get_interface_ip() helper to resolve an interface's IPv4 address via 'ip -o addr show'. Use it to back-propagate gateway into DHCP ranges so clients receive their default route. - set_dhcp_range() resolves gateway: explicit > existing range > iface IP - DnsToFirewallSync and FirewallToDhcpSync sync ensure gateways are set - Remove automatic masquerade toggle from DnsToFirewallSync - Fix dnsmasq lease file path to /var/lib/misc/dnsmasq.leases - Rename lease state field expires_at -> expires (ISO string) - Add 'ip -o addr show' to sudo whitelist |
||
|
mteehan
|
fb39af126a |
docs: update documentation and project structure
- Update AGENTS.md, README.md, and docs/* with revisions - Refactor lib/acme.py and lib/state.py - Add tests for acme module - Remove install.sh and restart-services.sh (moved to scripts/) - Normalize vendor files (acme.sh, htm.js) |
||
|
mteehan
|
8c13ad55ce |
Add update-vendor.sh symlink support, unify install.sh vendor flow
- update-vendor.sh now creates webui/vendor symlinks (htm.js) - install.sh calls update-vendor.sh after package install - Add vendor/.empty and webui/vendor/.empty as directory placeholders in git |
||
|
mteehan
|
348bbfbca6 | dhcp: track pending config changes with hash, update UI button | ||
|
mteehan
|
25a1943fce |
Optimize firewall state collection and improve daemon shutdown
- Replace per-zone --list-all calls with single --list-all-zones in _collect_firewall - Add _parse_all_zones_output() parser with rich rules/rich-rules normalization - Convert daemon shutdown to async with proper runner cleanup and socket unlink - Add TimeoutStopSec=15 to vacuum-walld.service for graceful stop - Fix exception handling in _collect_dnsmasq - Remove management badge from proxy path rows |
||
|
mteehan
|
835326311b |
Refactor nginx to path-based domain model with config migration
Replace the legacy top-level management key with a unified paths-based model. Each domain now contains a paths map where each entry defines its own backend, auth, headers, and flags (is_management, is_websocket). - Add _migrate_config() to auto-migrate legacy formats on first load - Remove set_management_proxy() and POST_NGINX_MANAGEMENT endpoint - Update server_block.conf template to iterate paths with per-location auth - Update daemon handler, API blueprint, state collector, and install script - Add server config generation tests for paths, WebSocket, auth inheritance - Update frontend proxy page to display per-path rows with flags |
||
|
mteehan
|
8feb56faf6 |
fix: ECC cert support, ACME deploy hook path, NAT detection, and account config fallback
- Add find_cert_dir() to resolve both RSA and ECC (domain_ecc/) cert dirs - Copy acme deploy hook to /deploy/ where acme.sh resolves it - _parse_account_conf checks both legacy .account.conf and declarative config - Skip public DNS check when all local IPs are private (NAT) - Improve check message strings for validity and expiry status - Support timezone-aware date formats in _days_until parsing - Filter out "no" SAN domains in cert listing - Bump frontend asset version cache keys - Fix DOMContentLoaded race condition in app.js boot - Fix spread operator in certs.js modal template |
||
|
mteehan
|
398831b6e2 |
Refactor ACME module and add cert issuance conflict handling
- Move acme.sh utilities (_run_acme, _find_acme, etc.) from lib/state to lib/acme - Rewrite _parse_list_output to support pipe, tab, and column-separated formats - Add ConflictError (409) to block issuing when cert already exists - Move _find_issuance helper to detect in-progress issuance per domain - Update issue_cert to check existing certs and return issuance status - Fix start_polling to accept event loop explicitly - Add sudoers entry for chown on vacuum-wall.conf - Extend systemd ReadWritePaths for /run/nginx.pid and /var/log/nginx - Update frontend to handle 'existing' issuance status |
||
|
mteehan
|
5ba0f31767 |
Add state management, WebSocket polling, html.js templating, and refactor pages
- lib/state.py: per-subsystem collectors with versioned state store - daemon/server.py: state refresh on request, batch routing updates - webui/static/hoover/html.js: new html tag template helper via htm.js - webui/static/hoover/websocket.js: real-time state change notifications - webui/static/hoover/vdom.js: VDOM improvements for keyed diff - All frontend pages refactored to use html templates - Add tests for state management and polling - Update docs and AGENTS.md |
||
|
mteehan
|
5025dfaf30 |
feat: add ACME account management with validation pipeline
- Register, view, and deactivate ACME accounts via API and UI - 16-check validation framework for certificate issuance readiness - DNS resolution, port, nginx, and firewall pre-flight checks - External IP detection with NAT support and fallback providers - Account card and settings modal in certificates page - Guard certificate issuance behind account registration - Update modal CSS to overlay-based approach - 1000+ lines of tests for validation and account handlers |
||
|
mteehan
|
318d7169f7 |
Switch networkctl parsing from text to JSON output
Replace fragile text-based parsing of ● 1: lo
Link File: n/a
Network File: n/a
State: carrier (unmanaged)
Online state: unknown
Type: loopback
Hardware Address: 00:00:00:00:00:00
MTU: 65536
QDisc: noqueue
IPv6 Address Generation Mode: eui64
Number of Queues (Tx/Rx): 1/1
Address: 127.0.0.1
::1
May 30 22:43:27 vacuum-wall systemd-networkd[315]: lo: Link UP
May 30 22:43:27 vacuum-wall systemd-networkd[315]: lo: Gained carrier
● 77: eth0
Link File: /usr/lib/systemd/network/99-default.link
Network File: /etc/systemd/network/eth0.network
State: routable (configured)
Online state: online
Type: ether
Kind: veth
Driver: veth
Hardware Address: 8e:63:52:6b:ea:e8
MTU: 1500 (min: 68, max: 65535)
QDisc: noqueue
IPv6 Address Generation Mode: eui64
Number of Queues (Tx/Rx): 8/8
Auto negotiation: no
Speed: 10Gbps
Duplex: full
Port: tp
Address: 192.168.1.5 (DHCPv4 via 192.168.1.1)
2600:4040:a6c1:4a00:8c63:52ff:fe6b:eae8
fe80::8c63:52ff:fe6b:eae8
Gateway: 192.168.1.1
fe80::3ebd:c5ff:fe2b:bd99
DNS: 192.168.1.1
2600:4040:a6c1:4a00::1
Search Domains: myfiosgateway.com
Activation Policy: up
Required For Online: yes
DHCPv4 Client ID: 8e:63:52:6b:ea:e8
DHCPv6 Client IAID: 0xf3d61521
DHCPv6 Client DUID: DUID-EN/Vendor:0000ab11b94215a519e8ca54
May 30 22:43:27 vacuum-wall systemd-networkd[315]: eth0: Link UP
May 30 22:43:27 vacuum-wall systemd-networkd[315]: eth0: Gained carrier
May 30 22:43:27 vacuum-wall systemd-networkd[315]: eth0: Configuring with /etc/systemd/network/eth0.network.
May 30 22:43:27 vacuum-wall systemd-networkd[315]: eth0: Gained IPv6LL
May 30 22:43:27 vacuum-wall systemd-networkd[315]: eth0: DHCPv4 address 192.168.1.5/24, gateway 192.168.1.1 acquired from 192.168.1.1
● 79: eth1
Link File: /usr/lib/systemd/network/99-default.link
Network File: /etc/systemd/network/50-eth1.network
State: routable (configured)
Online state: online
Type: ether
Kind: veth
Driver: veth
Hardware Address: 8e:63:52:6b:ea:8e
MTU: 1500 (min: 68, max: 65535)
QDisc: noqueue
IPv6 Address Generation Mode: eui64
Number of Queues (Tx/Rx): 8/8
Auto negotiation: no
Speed: 10Gbps
Duplex: full
Port: tp
Address: 10.4.20.1
fd42:a304:c836:2a7f:8c63:52ff:fe6b:ea8e
fe80::8c63:52ff:fe6b:ea8e
Gateway: fe80::1266:6aff:fe76:bc5b
DNS: fd42:a304:c836:2a7f::1
Activation Policy: up
Required For Online: yes
DHCPv6 Client IAID: 0x1da7c7a5
DHCPv6 Client DUID: DUID-EN/Vendor:0000ab11b94215a519e8ca54
Jun 01 03:48:43 vacuum-wall systemd-networkd[315]: eth1: Reconfiguring with /etc/systemd/network/50-eth1.network.
Jun 01 04:05:47 vacuum-wall systemd-networkd[315]: eth1: Reconfiguring with /etc/systemd/network/99-eth1.network.
Jun 01 04:05:47 vacuum-wall systemd-networkd[315]: eth1: DHCPv6 lease lost
Jun 01 04:05:47 vacuum-wall systemd-networkd[315]: eth1: Reconfiguring with /etc/systemd/network/99-eth1.network.
Jun 01 04:06:51 vacuum-wall systemd-networkd[315]: eth1: Reconfiguring with /etc/systemd/network/99-eth1.network.
Jun 01 04:06:51 vacuum-wall systemd-networkd[315]: eth1: DHCPv6 lease lost
Jun 01 04:06:51 vacuum-wall systemd-networkd[315]: eth1: Reconfiguring with /etc/systemd/network/99-eth1.network.
Jun 01 04:09:35 vacuum-wall systemd-networkd[315]: eth1: Reconfiguring with /etc/systemd/network/50-eth1.network.
Jun 01 04:09:35 vacuum-wall systemd-networkd[315]: eth1: DHCPv6 lease lost
Jun 01 04:09:35 vacuum-wall systemd-networkd[315]: eth1: Reconfiguring with /etc/systemd/network/50-eth1.network. with structured JSON parsing using {"Interfaces":[{"Index":1,"Name":"lo","Type":"loopback","Flags":65609,"FlagsString":"up,loopback,running,lower-up","KernelOperationalState":0,"KernelOperationalStateString":"unknown","MTU":65536,"MinimumMTU":0,"MaximumMTU":4294967295,"AdministrativeState":"unmanaged","OperationalState":"carrier","CarrierState":"carrier","AddressState":"off","IPv4AddressState":"off","IPv6AddressState":"off","OnlineState":null,"Addresses":[{"Family":2,"Address":[127,0,0,1],"PrefixLength":8,"ConfigSource":"foreign","Scope":254,"ScopeString":"host","Flags":128,"FlagsString":"permanent","ConfigState":"configured"},{"Family":10,"Address":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],"PrefixLength":128,"ConfigSource":"foreign","Scope":254,"ScopeString":"host","Flags":128,"FlagsString":"permanent","ConfigState":"configured"}],"Routes":[{"Family":10,"Destination":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1],"DestinationPrefixLength":128,"TOS":0,"Scope":0,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"global","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[127,255,255,255],"DestinationPrefixLength":32,"PreferredSource":[127,0,0,1],"TOS":0,"Scope":253,"Protocol":2,"Type":3,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"link","ProtocolString":"kernel","TypeString":"broadcast","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[127,0,0,1],"DestinationPrefixLength":32,"PreferredSource":[127,0,0,1],"TOS":0,"Scope":254,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"host","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[127,0,0,0],"DestinationPrefixLength":8,"PreferredSource":[127,0,0,1],"TOS":0,"Scope":254,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"host","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"}]},{"Index":77,"Name":"eth0","Kind":"veth","Type":"ether","Driver":"veth","Flags":69699,"FlagsString":"up,broadcast,running,multicast,lower-up","KernelOperationalState":6,"KernelOperationalStateString":"up","MTU":1500,"MinimumMTU":68,"MaximumMTU":65535,"HardwareAddress":[142,99,82,107,234,232],"BroadcastAddress":[255,255,255,255,255,255],"IPv6LinkLocalAddress":[254,128,0,0,0,0,0,0,140,99,82,255,254,107,234,232],"AdministrativeState":"configured","OperationalState":"routable","CarrierState":"carrier","AddressState":"routable","IPv4AddressState":"routable","IPv6AddressState":"routable","OnlineState":"online","NetworkFile":"/etc/systemd/network/eth0.network","NetworkFileDropins":[],"RequiredForOnline":true,"RequiredOperationalStateForOnline":[null,null],"RequiredFamilyForOnline":"any","ActivationPolicy":"up","DNS":[{"Family":2,"Address":[192,168,1,1],"ConfigSource":"DHCPv4","ConfigProvider":[192,168,1,1]},{"Family":10,"Address":[38,0,64,64,166,193,74,0,0,0,0,0,0,0,0,1],"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,62,189,197,255,254,43,189,153]}],"SearchDomains":[{"Domain":"myfiosgateway.com","ConfigSource":"DHCPv4","ConfigProvider":[192,168,1,1]}],"DNSSettings":[{"LLMNR":"yes","ConfigSource":"static"},{"MDNS":"no","ConfigSource":"static"}],"Addresses":[{"Family":2,"Address":[192,168,1,5],"PrefixLength":24,"ConfigSource":"DHCPv4","ConfigProvider":[192,168,1,1],"Broadcast":[192,168,1,255],"Scope":0,"ScopeString":"global","Flags":0,"FlagsString":null,"PreferredLifetimeUSec":1734964293205,"PreferredLifetimeUsec":1734964293205,"ValidLifetimeUSec":1734964293205,"ValidLifetimeUsec":1734964293205,"ConfigState":"configured"},{"Family":10,"Address":[254,128,0,0,0,0,0,0,140,99,82,255,254,107,234,232],"PrefixLength":64,"ConfigSource":"foreign","Scope":253,"ScopeString":"link","Flags":128,"FlagsString":"permanent","ConfigState":"configured"},{"Family":10,"Address":[38,0,64,64,166,193,74,0,140,99,82,255,254,107,234,232],"PrefixLength":64,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,62,189,197,255,254,43,189,153],"Scope":0,"ScopeString":"global","Flags":768,"FlagsString":"manage-temporary-address,no-prefixroute","PreferredLifetimeUSec":1677495569859,"PreferredLifetimeUsec":1677495569859,"ValidLifetimeUSec":1677495569859,"ValidLifetimeUsec":1677495569859,"ConfigState":"configured"}],"NextHops":[{"ID":1635324079,"Family":10,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,62,189,197,255,254,43,189,153],"Gateway":[254,128,0,0,0,0,0,0,62,189,197,255,254,43,189,153],"Flags":0,"FlagsString":"","Protocol":9,"ProtocolString":"9","Blackhole":false,"ConfigState":"configured"}],"Routes":[{"Family":2,"Destination":[192,168,1,0],"DestinationPrefixLength":24,"PreferredSource":[192,168,1,5],"TOS":0,"Scope":253,"Protocol":2,"Type":1,"Priority":1024,"Table":254,"Flags":0,"ConfigSource":"foreign","ScopeString":"link","ProtocolString":"kernel","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[192,168,1,255],"DestinationPrefixLength":32,"PreferredSource":[192,168,1,5],"TOS":0,"Scope":253,"Protocol":2,"Type":3,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"link","ProtocolString":"kernel","TypeString":"broadcast","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":10,"Destination":[38,0,64,64,166,193,74,0,140,99,82,255,254,107,234,232],"DestinationPrefixLength":128,"TOS":0,"Scope":0,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"global","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":10,"Destination":[38,0,64,64,166,193,74,0,0,0,0,0,0,0,0,0],"DestinationPrefixLength":64,"TOS":0,"Scope":0,"Protocol":9,"Type":1,"Priority":1024,"Table":254,"Flags":0,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,62,189,197,255,254,43,189,153],"ScopeString":"global","ProtocolString":"9","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","LifetimeUSec":1677495568830,"ConfigState":"configured"},{"Family":10,"Destination":[254,128,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"DestinationPrefixLength":64,"TOS":0,"Scope":0,"Protocol":2,"Type":1,"Priority":256,"Table":254,"Flags":0,"ConfigSource":"foreign","ScopeString":"global","ProtocolString":"kernel","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[0,0,0,0],"DestinationPrefixLength":0,"Gateway":[192,168,1,1],"PreferredSource":[192,168,1,5],"TOS":0,"Scope":0,"Protocol":16,"Type":1,"Priority":1024,"Table":254,"Flags":0,"ConfigSource":"DHCPv4","ConfigProvider":[192,168,1,1],"ScopeString":"global","ProtocolString":"16","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[192,168,1,1],"DestinationPrefixLength":32,"PreferredSource":[192,168,1,5],"TOS":0,"Scope":253,"Protocol":16,"Type":1,"Priority":1024,"Table":254,"Flags":0,"ConfigSource":"DHCPv4","ConfigProvider":[192,168,1,1],"ScopeString":"link","ProtocolString":"16","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[192,168,1,5],"DestinationPrefixLength":32,"PreferredSource":[192,168,1,5],"TOS":0,"Scope":254,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"host","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":10,"Destination":[255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"DestinationPrefixLength":8,"TOS":0,"Scope":0,"Protocol":2,"Type":5,"Priority":256,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"global","ProtocolString":"kernel","TypeString":"multicast","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":10,"Destination":[254,128,0,0,0,0,0,0,140,99,82,255,254,107,234,232],"DestinationPrefixLength":128,"TOS":0,"Scope":0,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"global","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"}],"DHCPv4Client":{"Lease":{"LeaseTimestampUSec":1648564292241,"Timeout1USec":1691764292241,"Timeout2USec":1724164292241},"ClientIdentifier":[1,142,99,82,107,234,232]},"DHCPv6Client":{"Lease":{"LeaseTimestampUSec":958440922609},"DUID":[0,2,0,0,171,17,185,66,21,165,25,232,202,84]}},{"Index":79,"Name":"eth1","Kind":"veth","Type":"ether","Driver":"veth","Flags":69699,"FlagsString":"up,broadcast,running,multicast,lower-up","KernelOperationalState":6,"KernelOperationalStateString":"up","MTU":1500,"MinimumMTU":68,"MaximumMTU":65535,"HardwareAddress":[142,99,82,107,234,142],"BroadcastAddress":[255,255,255,255,255,255],"IPv6LinkLocalAddress":[254,128,0,0,0,0,0,0,140,99,82,255,254,107,234,142],"AdministrativeState":"configured","OperationalState":"routable","CarrierState":"carrier","AddressState":"routable","IPv4AddressState":"routable","IPv6AddressState":"routable","OnlineState":"online","NetworkFile":"/etc/systemd/network/50-eth1.network","NetworkFileDropins":[],"RequiredForOnline":true,"RequiredOperationalStateForOnline":[null,null],"RequiredFamilyForOnline":"any","ActivationPolicy":"up","DNS":[{"Family":10,"Address":[253,66,163,4,200,54,42,127,0,0,0,0,0,0,0,1],"ConfigSource":"DHCPv6","ConfigProvider":[254,128,0,0,0,0,0,0,18,102,106,255,254,118,188,91]},{"Family":10,"Address":[253,66,163,4,200,54,42,127,0,0,0,0,0,0,0,1],"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,18,102,106,255,254,118,188,91]}],"DNSSettings":[{"LLMNR":"yes","ConfigSource":"static"},{"MDNS":"no","ConfigSource":"static"}],"Addresses":[{"Family":10,"Address":[254,128,0,0,0,0,0,0,140,99,82,255,254,107,234,142],"PrefixLength":64,"ConfigSource":"foreign","Scope":253,"ScopeString":"link","Flags":128,"FlagsString":"permanent","ConfigState":"configured"},{"Family":2,"Address":[10,4,20,1],"PrefixLength":24,"ConfigSource":"static","Broadcast":[10,4,20,255],"Scope":0,"ScopeString":"global","Flags":128,"FlagsString":"permanent","ConfigState":"configured"},{"Family":10,"Address":[253,66,163,4,200,54,42,127,140,99,82,255,254,107,234,142],"PrefixLength":64,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,18,102,106,255,254,118,188,91],"Scope":0,"ScopeString":"global","Flags":896,"FlagsString":"permanent,manage-temporary-address,no-prefixroute","ConfigState":"configured"}],"NextHops":[{"ID":3144860678,"Family":10,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,18,102,106,255,254,118,188,91],"Gateway":[254,128,0,0,0,0,0,0,18,102,106,255,254,118,188,91],"Flags":0,"FlagsString":"","Protocol":9,"ProtocolString":"9","Blackhole":false,"ConfigState":"configured"}],"Routes":[{"Family":2,"Destination":[10,4,20,255],"DestinationPrefixLength":32,"PreferredSource":[10,4,20,1],"TOS":0,"Scope":253,"Protocol":2,"Type":3,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"link","ProtocolString":"kernel","TypeString":"broadcast","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":10,"Destination":[254,128,0,0,0,0,0,0,140,99,82,255,254,107,234,142],"DestinationPrefixLength":128,"TOS":0,"Scope":0,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"global","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":10,"Destination":[253,66,163,4,200,54,42,127,0,0,0,0,0,0,0,0],"DestinationPrefixLength":64,"TOS":0,"Scope":0,"Protocol":9,"Type":1,"Priority":1024,"Table":254,"Flags":0,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,18,102,106,255,254,118,188,91],"ScopeString":"global","ProtocolString":"9","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":10,"Destination":[253,66,163,4,200,54,42,127,140,99,82,255,254,107,234,142],"DestinationPrefixLength":128,"TOS":0,"Scope":0,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"global","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[10,4,20,1],"DestinationPrefixLength":32,"PreferredSource":[10,4,20,1],"TOS":0,"Scope":254,"Protocol":2,"Type":2,"Priority":0,"Table":255,"Flags":0,"ConfigSource":"foreign","ScopeString":"host","ProtocolString":"kernel","TypeString":"local","TableString":"local","Preference":0,"FlagsString":"","ConfigState":"configured"},{"Family":2,"Destination":[10,4,20,0],"DestinationPrefixLength":24,"PreferredSource":[10,4,20,1],"TOS":0,"Scope":253,"Protocol":2,"Type":1,"Priority":0,"Table":254,"Flags":0,"ConfigSource":"foreign","ScopeString":"link","ProtocolString":"kernel","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","ConfigState":"configured"}],"DHCPv6Client":{"Lease":{"LeaseTimestampUSec":1585104388996},"DUID":[0,2,0,0,171,17,185,66,21,165,25,232,202,84]}}],"Routes":[{"Family":10,"Destination":[38,0,64,64,166,193,74,0,0,0,0,0,0,0,0,0],"DestinationPrefixLength":56,"TOS":0,"Scope":0,"Protocol":9,"Type":1,"Priority":512,"Table":254,"Flags":0,"NextHopID":1635324079,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,62,189,197,255,254,43,189,153],"ScopeString":"global","ProtocolString":"9","TypeString":"unicast","TableString":"main","Preference":1,"FlagsString":"","LifetimeUSec":1677495568830,"ConfigState":"configured"},{"Family":10,"Destination":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"DestinationPrefixLength":0,"TOS":0,"Scope":0,"Protocol":9,"Type":1,"Priority":1024,"Table":254,"Flags":0,"NextHopID":3144860678,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,18,102,106,255,254,118,188,91],"ScopeString":"global","ProtocolString":"9","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","LifetimeUSec":1671685709586,"ConfigState":"configured"},{"Family":10,"Destination":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"DestinationPrefixLength":0,"TOS":0,"Scope":0,"Protocol":9,"Type":1,"Priority":1024,"Table":254,"Flags":0,"NextHopID":1635324079,"ConfigSource":"NDisc","ConfigProvider":[254,128,0,0,0,0,0,0,62,189,197,255,254,43,189,153],"ScopeString":"global","ProtocolString":"9","TypeString":"unicast","TableString":"main","Preference":0,"FlagsString":"","LifetimeUSec":1671195568830,"ConfigState":"configured"}],"RoutingPolicyRules":[{"Family":10,"Protocol":2,"ProtocolString":"kernel","TOS":0,"Type":1,"TypeString":"table","IPProtocol":0,"IPProtocolString":"ip","Priority":0,"FirewallMark":0,"FirewallMask":0,"Table":255,"TableString":"local","Invert":false,"ConfigSource":"foreign","ConfigState":"configured"},{"Family":10,"Protocol":2,"ProtocolString":"kernel","TOS":0,"Type":1,"TypeString":"table","IPProtocol":0,"IPProtocolString":"ip","Priority":32766,"FirewallMark":0,"FirewallMask":0,"Table":254,"TableString":"main","Invert":false,"ConfigSource":"foreign","ConfigState":"configured"},{"Family":2,"Protocol":2,"ProtocolString":"kernel","TOS":0,"Type":1,"TypeString":"table","IPProtocol":0,"IPProtocolString":"ip","Priority":32767,"FirewallMark":0,"FirewallMask":0,"Table":253,"TableString":"default","Invert":false,"ConfigSource":"foreign","ConfigState":"configured"},{"Family":2,"Protocol":2,"ProtocolString":"kernel","TOS":0,"Type":1,"TypeString":"table","IPProtocol":0,"IPProtocolString":"ip","Priority":0,"FirewallMark":0,"FirewallMask":0,"Table":255,"TableString":"local","Invert":false,"ConfigSource":"foreign","ConfigState":"configured"},{"Family":2,"Protocol":2,"ProtocolString":"kernel","TOS":0,"Type":1,"TypeString":"table","IPProtocol":0,"IPProtocolString":"ip","Priority":32766,"FirewallMark":0,"FirewallMask":0,"Table":254,"TableString":"main","Invert":false,"ConfigSource":"foreign","ConfigState":"configured"}]}. This provides more reliable and maintainable runtime state extraction.
lib/network.py:
- Rewrite parse_networkctl_status() to parse JSON instead of text lines
- Add _bytes_to_ip() helper for converting address byte arrays to IP strings
- Extract addresses, gateway (from Routes), DNS, MAC, and state from structured JSON
- Add proper error handling for malformed JSON input
daemon/handlers/network.py:
- Update all 3 callers (get_interfaces, get_interface, save_interface) to use --json=short
- Fix get_interfaces to include runtime-only interfaces by unioning config and runtime names (minus lo), rather than only iterating config-defined interfaces
lib/state.py:
- Update _collect_networkd to use --json=short flag
tests/test_network.py, tests/test_network_integration.py:
- Update all test fixtures from text output to matching JSON structure
|
||
|
mteehan
|
c5813d68b3 | refactor: update lib modules (common, dnsmasq, logging, network, state, nginx) | ||
|
mteehan
|
bc72db903c |
feat: add networkd subsystem and fix code review issues
Phase 1-4: Networkd subsystem - lib/network.py: systemd-networkd config renderer (.network INI files) with full schema support: [Match], [Link], [Network], [Address], [Route], [DHCPv4], [DHCPv6] sections. One Address/=DNS= line per value per spec. Route sections use #N suffix per systemd.syntax(7). - lib/network.py: generate_network_files() with 50-<name>.network prefix and stale file cleanup - lib/network.py: collect_upstream_dns() filters local/private DNS - lib/network.py: infer_dhcp_ranges() and infer_zones() helpers - daemon/handlers/network.py: routes for GET/POST /network/interfaces and full apply with DNS upstream sync to dnsmasq - webui/api/network.py: Flask blueprint for /api/network/* endpoints - webui/api: interfaces page updated with IP config inline editing - lib/state.py: networkd collector using parse_networkctl_status() - system/sudoers.d/vacuum-walld: networkctl + systemd-network rules - system/systemd/vacuum-walld.service: ReadWritePaths for /etc/systemd/network - install.sh: ACME email now optional, configured from WebUI - lib/acme.py: get_email() falls back to declarative config Phase 5: Code review fixes - daemon/server.py: path params now win over JSON body and query params in request body merge (prevents config save name override) - daemon/server.py: remove dead 'import re' - daemon/handlers/network.py: replace Path.mkdir() with sudo mkdir for /etc/systemd/network (ProtectSystem=strict compatibility) - system/sudoers.d/vacuum-walld: pin systemctl to specific commands (reload/is-active dnsmasq instead of wildcard) - system/sudoers.d/vacuum-walld: restore !requiretty and section comment - lib/network.py: remove unused _MANAGEMENT_PORTS constant - webui/api/network.py: remove redundant body[\name\] = name in save_interface Tests: 332 passing (110 new/updated), ruff clean |
||
|
mteehan
|
2f215793e9 |
docs: add docstrings to all API endpoints and daemon handlers
Add comprehensive docstrings to firewall, DHCP, proxy, wireguard, certs, and logs API endpoints. Document parameters, return values, and error cases for the documentation system. |
||
|
mteehan
|
dc96e15643 |
feat: pre-computed state store and async ACME issuance (fixes timeout mismatch)
- Add lib/state.py: in-memory state store with subsystem collectors (firewall, dnsmasq, nginx, acme, wireguard) - Refactor all handlers: read from state on GET, call refresh_state() after mutations instead of invoking subprocesses per request - daemon/server.py: add refresh_state(), /status/all, /status/refresh; populate state at startup - webui/api/certs.py: async step-by-step ACME issuance (validate, issue with request_id, poll status) replacing blocking endpoint - webui/server.py: render pages from state instead of direct lib calls - Update templates, JS for async cert issuance with polling UI - Update tests for state-based mocking; add test_state.py - Fix SIM105 lint issue (contextlib.suppress) - Add TODO.md with certificate issuance issue tracking Resolves: WebUI 30s timeout freeze during cert issuance (Problem 1) |