docs: update documentation and project structure
- Update AGENTS.md, README.md, and docs/* with revisions - Refactor lib/acme.py and lib/state.py - Add tests for acme module - Remove install.sh and restart-services.sh (moved to scripts/) - Normalize vendor files (acme.sh, htm.js)
This commit is contained in:
@@ -24,74 +24,72 @@ through the daemon client over a Unix socket.
|
||||
|
||||
### Code Layout
|
||||
|
||||
- `webui/server.py` — Flask app entry point. **Only** file that creates the `app`. SPA catch-all renders `index.html` with server-side `__WS_URL_PLACEHOLDER__` substitution (no Jinja).
|
||||
- `webui/server.py` — Flask app entry point. **Only** file that creates the `app`. SPA root route (`/`) renders `index.html` with server-side `__WS_URL_PLACEHOLDER__` substitution (no Jinja). All other paths return 404.
|
||||
- `webui/api/*.py` — Flask blueprints, one per subsystem. Routes prefix `/api/<subsystem>/`. All call `daemon.client` instead of `lib/` directly.
|
||||
- `webui/api/common.py` — Shared `_ok()` / `_error()` response helpers used by all blueprints.
|
||||
- `daemon/server.py` — aiohttp server, route registry, batch routing, WebSocket broadcast, state refresh.
|
||||
- `daemon/server.py` — aiohttp server, route registry, batch routing, WebSocket broadcast, state refresh, periodic polling.
|
||||
- `daemon/client.py` — Sync HTTP client over Unix socket using `requests_unixsocket.Session`.
|
||||
- `daemon/iface.py` — **Single source of truth** for all daemon API endpoints. Every endpoint is a frozen `(method, path)` tuple. Renaming an endpoint here auto-updates both server registry and client calls. All blueprints and handlers import from here.
|
||||
- `daemon/handlers/*.py` — Privileged operation handlers (all `sudo` calls live here).
|
||||
- `daemon/iface.py` — **Single source of truth** for all daemon API endpoints. Every endpoint is a frozen `(method, path)` tuple. Renaming here auto-updates both server registry and client calls.
|
||||
- `daemon/handlers/*.py` — Privileged operation handlers. All `sudo` calls live here.
|
||||
- `lib/state.py` — In-memory state store with per-subsystem collectors. Populated at daemon startup, refreshed on request. Backs WebSocket versioning/broadcast.
|
||||
- `lib/common.py` — Shared utilities: `run()`, `run_proc()`, `load_json()`, `save_json()`, `deep_merge()`, `ensure_dirs()`, `validate_interface_name()`. All `lib/` modules use these instead of defining local helpers.
|
||||
- `lib/logging.py` — Logging setup used by both webui and daemon.
|
||||
- `lib/common.py` — Shared utilities: `run()`, `run_proc()`, `load_json()`, `save_json()`, `deep_merge()`, `ensure_dirs()`, `config_hash()`, `validate_interface_name()`.
|
||||
- `lib/logging.py` — Logging setup used by both webui and daemon. Reads `VACUUM_WALL_LOG_LEVEL`.
|
||||
- `lib/*.py` — Backend modules (parsing, config, shared logic). Full type hints and `__all__` exports. No sudo calls.
|
||||
- `vendor/` — Vendored scripts and JS libraries (`acme.sh`, `htm`).
|
||||
- `data/` — Runtime artifacts (generated .confs, `.htpasswd`, ACME certs, firewall backup, dnsmasq fragments).
|
||||
- `config/<subsystem>/config.json` — Declarative JSON configs (source of truth). Generated `.conf` in `data/nginx/sites-enabled/`. Certs in `data/acme/`.
|
||||
- `config/<subsystem>/config.json` — Declarative JSON configs (source of truth).
|
||||
- `system/` — System file templates. `systemd/` (units installed to `/etc/systemd/system/`), `sudoers.d/`, `nginx/`.
|
||||
|
||||
Project uses `.venv`. Install deps with `pip install -e .` (from `pyproject.toml`). `__init__.py` files in `webui/`, `lib/`, and `daemon/` are intentionally empty.
|
||||
Project uses `.venv`. Install deps with `pip install -e .` (from `pyproject.toml`).
|
||||
`__init__.py` files in `webui/`, `lib/`, and `daemon/` are intentionally empty.
|
||||
|
||||
### Frontend (hoover)
|
||||
|
||||
Custom reactive SPA framework at `webui/static/hoover/`. See `docs/hoover.md` for full API reference.
|
||||
|
||||
Conventions:
|
||||
- All imports from `/static/hoover/index.js` (barrel export of reactivity, VDOM, router, API, components).
|
||||
- Pages in `webui/static/pages/` export `definePage({ init, subscribe, load, render })` as default.
|
||||
- All imports from `/static/hoover/index.js` (barrel export).
|
||||
- Pages in `webui/static/pages/` export `definePage({ init, subscribe, load, render })`.
|
||||
- Bootstrap: `webui/static/app.js` mounts two render roots (`#sidebar`, `#main`), then `connect()` for WS.
|
||||
- `h()` builds VNodes; `html` tag (from htm) enables JSX-like templates; `#comp` + `hComp()` for component lifecycle; `key` for keyed diff.
|
||||
- Events: `h()` uses `on:click` prefix. `html` templates use camelCase `onClick` (adapter translates to `on:click`).
|
||||
- `h()` builds VNodes with `on:click` prefix. `html` tag (htm) templates use camelCase `onClick` (adapter translates).
|
||||
- State always has `loading`, `refreshing`, `error` plus data. `load()` receives `(state, abortController, entry)`.
|
||||
- `openModal` + `formModal` for dialogs; `apiSubmit()` for form submission. `ToastContainer()` in main root.
|
||||
- `openModal` + `formModal` for dialogs; `apiSubmit()` for form submission.
|
||||
- No build step — ES modules served raw. Assets versioned via `?v=N` query string.
|
||||
|
||||
### Daemon Endpoints
|
||||
|
||||
- Unix socket at `data/daemon.sock` (configurable via `VACUUM_WALLD_SOCKET` env var)
|
||||
- WebSocket at `127.0.0.1:9091` (configurable via `VACUUM_WALLD_WS_PORT`) for real-time state change notifications
|
||||
- Can be started as `python -m daemon.server` or via the `vacuum-walld` console script
|
||||
- Unix socket at `data/daemon.sock` (configurable via `VACUUM_WALLD_SOCKET`)
|
||||
- WebSocket at `127.0.0.1:9091` (configurable via `VACUUM_WALLD_WS_PORT`) for real-time state notifications
|
||||
- Periodic polling per subsystem via `lib.state._DEFAULT_POLL_INTERVALS`, overridable with `VACUUM_WALL_POLL_INTERVALS` env var (format `subsystem:seconds,subsystem:seconds`)
|
||||
- Start as `python -m daemon.server` or via the `vacuum-walld` console script
|
||||
|
||||
## Environment Variables
|
||||
|
||||
- `VACUUM_WALL_DEV` — dev mode flag; when set, disables aggressive static asset caching
|
||||
- `VACUUM_WALLD_SOCKET` — override daemon socket path
|
||||
- `VACUUM_WALL_DEV` — dev mode flag; disables aggressive static asset caching
|
||||
- `VACUUM_WALL_LOG_LEVEL` — log level (default `INFO`)
|
||||
- `VACUUM_WALLD_SOCKET` — override daemon socket path (default `data/daemon.sock`)
|
||||
- `VACUUM_WALLD_WS_PORT` — override WebSocket port (default `9091`)
|
||||
|
||||
## Deployment
|
||||
|
||||
`install.sh` installs only system components and configures them; the project serves from the repo root by default. Options via env vars or CLI flags (CLI takes precedence). Set `INSTALL_DIR` or `--path` to override. Use `--dev` to auto-detect repo owner as service user; non-dev requires `--user`.
|
||||
|
||||
All Python modules use `Path(__file__).resolve().parent.parent` for `PROJECT_DIR` — no hardcoded paths. ACME certs at `PROJECT_DIR/data/acme/`.
|
||||
|
||||
### Service Start Order
|
||||
|
||||
`firewalld` → `avahi-daemon` → `dnsmasq` → `vacuum-walld` → `vacuum-wall`
|
||||
- `VACUUM_WALL_POLL_INTERVALS` — override poll intervals, e.g. `firewall:60,wireguard:5`
|
||||
- `VACUUM_WALL_EXTERNAL_IP_URL` — custom URL for external IP detection (acme handler)
|
||||
|
||||
## Local Dev
|
||||
|
||||
```bash
|
||||
.venv/bin/python webui/server.py # binds 127.0.0.1:9090
|
||||
# Setup
|
||||
python3 -m venv .venv && . .venv/bin/activate
|
||||
pip install -e ".[dev]"
|
||||
bash scripts/update-vendor.sh # fetches acme.sh + htm.js
|
||||
|
||||
# Start (Flask only, binds 127.0.0.1:9090)
|
||||
.venv/bin/python webui/server.py
|
||||
```
|
||||
|
||||
Reload running Flask via SIGHUP (auto-reloads `webui.*` and `lib.*` modules, then SIGTERM restart).
|
||||
|
||||
In production: systemd units run with `NoNewPrivileges`, `ProtectSystem=strict`, loopback-only networking.
|
||||
## Blueprint / Handler / lib Mapping
|
||||
|
||||
## Blueprint ↔ Handler ↔ lib Mapping
|
||||
|
||||
| Blueprint | URL Prefix | Handler Module | lib Module |
|
||||
|-----------------------|---------------------|--------------------------|-----------------|
|
||||
| Blueprint | URL Prefix | Handler | lib Module |
|
||||
|-----------------------|---------------------|----------------------------|-----------------|
|
||||
| `webui/api/firewall` | `/api/firewall/` | `daemon/handlers/firewall` | `lib.firewall` |
|
||||
| `webui/api/dhcp` | `/api/dhcp/` | `daemon/handlers/dnsmasq` | `lib.dnsmasq` |
|
||||
| `webui/api/proxy` | `/api/proxy/` | `daemon/handlers/nginx` | `lib.nginx` |
|
||||
@@ -99,6 +97,7 @@ In production: systemd units run with `NoNewPrivileges`, `ProtectSystem=strict`,
|
||||
| `webui/api/wireguard` | `/api/wireguard/` | `daemon/handlers/wireguard`| `lib.wireguard` |
|
||||
| `webui/api/network` | `/api/network/` | `daemon/handlers/network` | `lib.network` |
|
||||
| `webui/api/logs` | `/api/logs/` | `daemon/handlers/logs` | — |
|
||||
| `webui/api/status` | `/api/status/` | `daemon/handlers/status` | — |
|
||||
|
||||
## Privileged Operations
|
||||
|
||||
@@ -111,20 +110,25 @@ Adding a new privileged command requires a sudoers entry **and** the `daemon/han
|
||||
|
||||
## API Response Contract
|
||||
|
||||
- Success: `{"ok": true, "data": <value>}` — helper `_ok(data)` from `webui.api.common` (Flask) or `ok(data)` from `daemon.server` (aiohttp).
|
||||
- Error: `{"ok": false, "error": "msg"}` — helper `_error(msg, code=400)` from `webui.api.common` or `error(msg, code)` from `daemon.server`.
|
||||
- `acme.issue()` / `acme.renew()` raise `RuntimeError` on failure — API layer wraps in try/except.
|
||||
- HTTP codes: `400` bad request, `404` not found, `500` internal failure.
|
||||
- Success: `{"ok": true, "data": <value>}` — `_ok(data)` (Flask) or `ok(data)` (aiohttp)
|
||||
- Error: `{"ok": false, "error": "msg"}` — `_error(msg, code)` (Flask) or `error(msg, code)` (aiohttp)
|
||||
- `acme.issue()` / `acme.renew()` raise `RuntimeError` on failure — API layer wraps in try/except
|
||||
- HTTP codes: `400` bad request, `404` not found, `409` conflict, `500` internal failure
|
||||
|
||||
## Deploy
|
||||
|
||||
`install.sh` is the single deploy script. Run as root. Only `MGMT_PASS` is strictly required; `MGMT_DOMAIN` is auto-detected from hostname.
|
||||
`scripts/install.sh` is the single deploy script. Run as root. CLI flags take precedence over env vars.
|
||||
`MGMT_PASS` is strictly required; `MGMT_DOMAIN` auto-detected from hostname.
|
||||
|
||||
### Service Start Order
|
||||
|
||||
`firewalld` → `avahi-daemon` → `dnsmasq` → `vacuum-walld` → `vacuum-wall`
|
||||
|
||||
## Lint and Tests
|
||||
|
||||
**Linter / formatter:** Ruff (`ruff check` + `ruff format`). Config in `pyproject.toml` under `[tool.ruff]`.
|
||||
**Linter / formatter:** Ruff (`ruff check` + `ruff format`). Config in `pyproject.toml`.
|
||||
|
||||
**Tests:** pytest in `tests/`. Tests mock out subprocess calls — no system services required.
|
||||
**Tests:** pytest in `tests/` (17 modules). All subprocess calls are mocked — no system services required.
|
||||
|
||||
```bash
|
||||
.venv/bin/ruff check lib/ webui/ tests/ # lint
|
||||
@@ -132,24 +136,16 @@ Adding a new privileged command requires a sudoers entry **and** the `daemon/han
|
||||
.venv/bin/python -m pytest tests/ -v # test
|
||||
```
|
||||
|
||||
Install dev tooling with `pip install -e ".[dev]"`.
|
||||
|
||||
## Docs
|
||||
|
||||
`docs/` contains the authoritative reference for each subsystem. **Before reasoning about any subsystem**, read the relevant doc(s) below.
|
||||
`docs/` contains the authoritative reference for each subsystem. Read the relevant doc before reasoning about a subsystem.
|
||||
|
||||
| Doc | Contents |
|
||||
|-----|----------|
|
||||
| `docs/architecture.md` | Request flow, subsystem communication, two-user model, zone model, state management |
|
||||
| `docs/security.md` | Privilege model, sudo whitelist, systemd hardening, TLS config, zone trust levels |
|
||||
| `docs/deployment.md` | Install script options, what install.sh does, post-install setup, troubleshooting |
|
||||
| `docs/deployment.md` | Install script options, what scripts/install.sh does, post-install setup, troubleshooting |
|
||||
| `docs/config.md` | JSON schema for each subsystem config (dnsmasq, nginx, wireguard, cert types) |
|
||||
| `docs/api.md` | REST API endpoint reference, request/response contracts, route patterns |
|
||||
| `docs/hoover.md` | Custom frontend framework API reference |
|
||||
| `docs/overview.md` | Subsystem summaries, tech stack, complete project directory tree |
|
||||
|
||||
## Important Rules
|
||||
1. Ask, don't assume. If something is unclear, ask before writing a single line.
|
||||
2. Simplest solution first. Always implement the simplest thing that could work.
|
||||
3. Don't touch unrelated code. If a file or function is not directly part of the current task, do not modify it.
|
||||
4. Flag uncertainty explicitly. If you are not confident about an approach or technical detail, say so.
|
||||
| `docs/overview.md` | Subsystem summaries, tech stack, complete project directory tree |
|
||||
Reference in New Issue
Block a user