refactor: unify project structure, improve security, and enhance deployment

- Fix WireGuard private key leak in API responses and config updates
- Update systemd service to serve from repo root with adjusted sandbox
- Add CLI flags, idempotency, and dev mode to install.sh
- Extract common utilities to lib/common.py and webui/api/common.py
- Migrate frontend to htmx for simpler, more maintainable UI
- Update docs to reflect current architecture and deployment model
- Vendor htmx dependencies per project requirements
This commit is contained in:
2026-05-25 00:53:32 +00:00
parent 8829ac579d
commit d1ab717c0f
36 changed files with 857 additions and 626 deletions
+27 -2
View File
@@ -103,9 +103,16 @@ Compare declarative config against live firewalld state. Returns diff for interf
PATCH /api/firewall/config
```
Deep-merge the provided fields into the existing config.
Deep-merge the provided fields into the existing config. Returns pending changes summary.
**Response:** `data` is `null` on success.
**Response (`data`):**
| Field | Type | Description |
|-------|------|-------------|
| `config_saved` | `boolean` | Always `true` |
| `pending` | `[object, ...]` | List of pending changes |
| `needs_apply` | `boolean` | Whether changes need to be applied |
| `unmanaged_zones` | `object` | Zones active on system but not in config |
### Zone Management
@@ -927,6 +934,24 @@ Replace the entire WireGuard configuration. The `private_key` field is stripped
**Response:** `data` contains the updated configuration (`private_key` omitted).
---
#### Partial Update Configuration
```
PATCH /api/wireguard/config
```
Deep-merge the provided fields into the existing configuration.
**Request Body:**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| *(any subset)* | `any` | Yes | Fields to merge into the existing config |
**Response:** `data` is `null` on success.
### Tunnel Control
#### Apply Configuration
+4 -4
View File
@@ -37,18 +37,18 @@ Flask WebUI ──→ lib/firewall.py ──→ sudo firewall-cmd ──→ fire
Flask WebUI ──→ lib/nginx.py ──→ write local .conf files ──→ sudo cp to /etc/nginx/ ──→ sudo nginx -t && sudo nginx -s reload
Flask WebUI ──→ lib/dnsmasq.py ──→ render config ──→ sudo tee /etc/dnsmasq.d/vacuum-wall.conf ──→ sudo systemctl reload dnsmasq
Flask WebUI ──→ lib/acme.py ──→ acme.sh (no sudo, runs as service user) ──→ ZeroSSL ACME
Flask WebUI ──→ lib/wireguard.py ──→ render $PROJECT_DIR/data/wireguard/wg0.conf ──→ sudo cp to /etc/wireguard/ ──→ sudo wg-quick up wg0
Flask WebUI ──→ lib/wireguard.py ──→ render data/wireguard/wg0.conf ──→ sudo cp to /etc/wireguard/ ──→ sudo wg-quick up wg0
```
Each `lib/` module encapsulates command construction, privilege escalation (via sudo where needed), and error handling for its subsystem. The modules read declarative configuration from `config/` and runtime artifacts from `data/`, render the appropriate system configuration files, and invoke the corresponding privileged operation. Note that `lib/acme.py` runs `acme.sh` without sudo — it executes as the unprivileged service user using webroot validation rather than standalone/TLS-ALPN modes that would require elevated privileges.
The `lib/` modules auto-discover the project root at runtime via `Path(__file__).resolve().parent.parent`, so no hardcoded paths are needed in Python code.
The `lib/` modules auto-discover the project root at runtime via `Path(__file__).resolve().parent.parent`. This works because `install.sh` performs an editable pip install (`pip install -e .`), keeping module files in the project directory rather than copying them to `site-packages/`.
## Install-Time Templating
System configuration files in `system/` are Jinja2 templates rendered by `install.sh` at install time:
- **`systemd/vacuum-wall.service`**, **`systemd/vacuum-wall-acme.service`** — `{{ USER_NAME }}`, `{{ PROJECT_DIR }}`, `{{ ACME_HOME }}` are substituted to produce the final systemd unit files installed to `/etc/systemd/system/`.
- **`systemd/vacuum-wall.service`**, **`systemd/vacuum-wall-acme.service`** — `{{ USER_NAME }}`, `{{ PROJECT_DIR }}`, `{{ ACME_HOME }}` are substituted to produce the final systemd unit files installed to `/etc/systemd/system/`. The `PROJECT_DIR` template variable is set from the `INSTALL_DIR` environment variable (defaults to the repo root).
- **`sudoers.d/vacuum-wall`** — `{{ USER_NAME }}` is substituted to produce the sudoers whitelist.
- The timer file (`vacuum-wall-acme.timer`) contains no variable paths and is installed as-is.
@@ -99,7 +99,7 @@ data/
└── wireguard/ # WireGuard runtime artifacts
```
Both `config/` and `data/` reside within the project directory (`$PROJECT_DIR/`). The systemd service unit's `ReadWritePaths` directive grants the Flask process write access to both directories, while keeping the rest of the filesystem read-only. The `PROJECT_DIR` value is templated into the service unit at install time.
Both `config/` and `data/` reside within the project directory. The systemd service unit's `ReadWritePaths` directive grants the Flask process write access to both directories, while keeping the rest of the filesystem read-only. The `INSTALL_DIR` value is templated into the service unit at install time.
## File System Layout
+3 -3
View File
@@ -107,7 +107,7 @@ This file defines reverse proxy domains, the management interface, and global SS
},
"auth": {
"user": "admin",
"htpasswd": "$PROJECT_DIR/data/nginx/.htpasswd"
"htpasswd": "data/nginx/.htpasswd"
}
},
"ssl": {
@@ -142,7 +142,7 @@ The `domains` object maps domain names (keys) to proxy configurations. Each entr
|---|---|
| `acme` | Vacuum Wall uses acme.sh to request and renew an ACME certificate via the HTTP-01 challenge. The nginx configuration is temporarily modified to serve the ACME challenge files at `/.well-known/acme-challenge/`. The `email` field is required. |
| `file` | Use a pre-existing certificate and private key from the local file system. The `path` and `key_path` fields must point to readable PEM files. Vacuum Wall will not attempt to renew these certificates. |
| `selfsigned` | Vacuum Wall generates a self-signed certificate and private key on first apply. Useful for internal domains or testing. The generated certificate is stored at `$PROJECT_DIR/data/certs/`. |
| `selfsigned` | Vacuum Wall generates a self-signed certificate and private key on first apply. Useful for internal domains or testing. The generated certificate is stored at `data/certs/`. |
### Management Domain
@@ -159,7 +159,7 @@ The `management` block configures the Vacuum Wall admin interface itself. It fol
The `.htpasswd` file can be created with the `htpasswd` utility:
```bash
htpasswd -bc $PROJECT_DIR/data/nginx/.htpasswd admin yourpassword
htpasswd -bc data/nginx/.htpasswd admin yourpassword
```
### Global SSL Settings
+56 -42
View File
@@ -19,54 +19,55 @@ This guide walks through deploying Vacuum Wall on a real appliance or server. Va
## Installation
Download the Vacuum Wall repository onto the target machine, then run the installer with the required environment variables:
Download the Vacuum Wall repository onto the target machine, then run the installer with required settings. All options accept both CLI flags and environment variables (CLI takes precedence).
```bash
# Option A: Public DNS
PROJECT_DIR="/opt/vacuum-wall" \
USER_NAME="vacuum-wall" \
# Production: all env vars
MGMT_DOMAIN=wall.example.com \
MGMT_PASS="strongpassword" \
MGMT_USER="admin" \
ACME_EMAIL="admin@example.com" \
bash install.sh
# Option B: mDNS (LAN-only, no DNS record needed)
MGMT_DOMAIN=vacuum-wall.local \
MGMT_PASS="strongpassword" \
MGMT_USER="admin" \
ACME_EMAIL="admin@example.com" \
bash install.sh
# Dev mode: CLI flags, auto-detects repo owner
./install.sh --dev --mgmt-pass strongpassword --acme-email "admin@example.com"
# mDNS (LAN-only, no DNS record needed)
./install.sh --mgmt-domain vacuum-wall.local --mgmt-pass strongpass --acme-email "me@example.com"
```
### Environment Variables
### Options
| Variable | Required | Description |
|---|---|---|
| `PROJECT_DIR` | No | Directory where the project resides. Auto-discovers from `install.sh` location if not set. |
| `USER_NAME` | No | System user that runs the WebUI service. Defaults to `vacuum-wall`. |
| `MGMT_DOMAIN` | No | The domain for the management WebUI. Defaults to `$hostname.local` (auto-detected from the system hostname), which works with mDNS on your LAN (avahi-daemon is installed and enabled automatically). Set explicitly for a custom DNS domain (e.g., `wall.example.com`). **Errors if hostname is undetectable and this var is not set.** |
| `MGMT_PASS` | Yes | The password for HTTP basic auth protecting the WebUI. Use a strong, randomly generated password. |
| `MGMT_USER` | No | The username for WebUI access. Defaults to `admin`. |
| `ACME_EMAIL` | Yes | The email address registered with the ACME provider (ZeroSSL by default) for certificate issuance and expiry notifications. |
All settings that can be passed as an environment variable also have a CLI flag equivalent. CLI flags take precedence over environment variables.
| Flag | Env Var | Required | Description |
|---|---|---|---|
| -- | `MGMT_DOMAIN` | No | Domain for the management WebUI. Defaults to `$hostname.local` (mDNS). Auto-detected from system hostname. **Errors if hostname is undetectable and this is not set.** |
| `--mgmt-domain` | `MGMT_DOMAIN` | No | (same as above) |
| `--mgmt-pass` | `MGMT_PASS` | Yes | Password for HTTP basic auth protecting the WebUI. |
| `--mgmt-user` | `MGMT_USER` | No | Username for WebUI access. Defaults to `admin`. |
| `--acme-email` | `ACME_EMAIL` | Yes | Email for ACME provider (ZeroSSL by default). |
| `--user, -u` | `USER_NAME` | No | System user for the WebUI service. Defaults to `vacuum-wall`. |
| `--path, -p` | `INSTALL_DIR` | No | Install directory. Defaults to repo root. Set to deploy from a custom path (e.g., `/opt/vacuum-wall`). |
| `--dev` | -- | No | Development mode: auto-detects repo owner as service user, skips safety warning. |
| `--wan-iface` | `WAN_IFACE` | No | WAN interface name. Auto-detected from default gateway. |
| `--lan-ifaces` | `LAN_IFACES` | No | LAN interface names, comma-separated. Auto-detected from non-loopback, non-WAN interfaces. |
Run `./install.sh --help` for full usage.
---
## Container / Custom Deployment
You can deploy Vacuum Wall in a container or at any custom path. Set `PROJECT_DIR` to the mount or bind path, and `USER_NAME` to whatever system user exists in the container or host environment:
You can deploy Vacuum Wall in a container or at any custom path. Use `--path` (or `INSTALL_DIR`) for the mount or bind path, and `--user` (or `USER_NAME`) for whatever system user exists:
```bash
# Docker volume mount example
PROJECT_DIR="/app/vacuum-wall" \
USER_NAME="ww-app" \
MGMT_DOMAIN="proxy.internal" \
MGMT_PASS="strongpassword" \
ACME_EMAIL="admin@example.com" \
bash install.sh
./install.sh --path /app/vacuum-wall --user ww-app \
--mgmt-domain proxy.internal --mgmt-pass strongpassword \
--acme-email "admin@example.com"
```
The systemd service unit files and sudoers whitelist are rendered from Jinja2 templates at install time, substituting `USER_NAME` and `PROJECT_DIR`. This means no hardcoded paths remain after installation.
The systemd service unit files and sudoers whitelist are rendered from Jinja2 templates at install time, substituting `USER_NAME` and `INSTALL_DIR`. This means no hardcoded paths remain after installation.
---
@@ -76,19 +77,20 @@ The installer performs the following steps automatically:
- **Package installation**: Installs firewalld, nginx, dnsmasq, avahi-daemon, wireguard-tools, python3, Flask, pip, jq, curl, iptables, nftables, and apache2-utils.
- **System user creation**: Creates a dedicated system user (default: `vacuum-wall`, configurable via `USER_NAME`) with a nologin shell that owns the project data and runs the WebUI service.
- **Python venv**: Sets up a Python virtual environment and installs project dependencies.
- **acme.sh installation**: Downloads and installs the acme.sh client to the project user's home directory for ACME certificate management.
- **Directory setup**: Creates config directories under `$PROJECT_DIR/config/` for each subsystem's declarative JSON, and data directories under `$PROJECT_DIR/data/` for nginx sites, dnsmasq fragments, firewall rules, and WireGuard config. Sets ownership to the configured system user.
- **Template rendering**: Renders system template files (`systemd/*.service`, `sudoers.d/`) via Jinja2, substituting `USER_NAME`, `PROJECT_DIR`, and `ACME_HOME`. Installed systemd and sudoers files contain no hardcoded values.
- **Python venv**: Creates or recreates the Python virtual environment and installs project dependencies.
- **acme.sh installation**: Copies the vendored acme.sh client to the data directory for ACME certificate management. Skips if already installed.
- **Directory setup**: Creates config directories under `config/` for each subsystem's declarative JSON, and data directories under `data/` for generated files (nginx sites, dnsmasq fragments, firewall backup, WireGuard config).
- **Template rendering**: Renders system template files (`systemd/*.service`, `sudoers.d/`) via Jinja2, substituting `USER_NAME`, `INSTALL_DIR`, and `ACME_HOME`. Installed systemd and sudoers files contain no hardcoded values.
- **Sudoers whitelist**: Installs a restrictive sudoers file at `/etc/sudoers.d/vacuum-wall` allowing the configured user to run only the specific privileged commands needed for firewall, nginx, and dnsmasq management. Validates syntax with `visudo -cf`.
- **IP forwarding**: Enables `net.ipv4.ip_forward=1` in sysctl.conf and applies it at runtime, required for routing traffic between zones.
- **IP forwarding**: Enables `net.ipv4.ip_forward=1` in sysctl.conf and applies it at runtime, required for routing traffic between zones. Appends only if not already present.
- **Firewalld initialization**: Starts and enables firewalld. Opens HTTP, HTTPS, and SSH services on the public zone for management access.
- **Dnsmasq initialization**: Starts and enables dnsmasq for future DHCP/DNS serving on internal interfaces.
- **mDNS broadcast**: Enables and starts avahi-daemon so the appliance advertises its hostname (`<hostname>.local`) on the local network.
- **Self-signed certificate**: Generates a temporary self-signed X.509 certificate for the management domain with the correct CN and SAN, placed where acme.sh would store a real cert.
- **Self-signed certificate**: Generates a temporary self-signed X.509 certificate for the management domain with the correct CN and SAN, placed where acme.sh would store a real cert. Skips if a certificate already exists (preserves real ACME certs).
- **Management proxy configuration**: Configures nginx as a reverse proxy that forward-proxies to the WebUI at `127.0.0.1:9090`, with HTTP-to-HTTPS redirect, basic auth, and WebSocket upgrade support.
- **Credentials**: Generates an htpasswd file using `apache2-utils` (with a Python fallback) for the management proxy's basic auth. Copies it to both `$USER_HOME/.htpasswd` (used by install.sh's initial nginx config) and `$PROJECT_DIR/data/nginx/.htpasswd` (used by the running app).
- **Initial nginx config**: Writes `$PROJECT_DIR/config/nginx/config.json` with the management domain and auth settings pre-configured, so the WebUI can render management proxy config out of the box.
- **Credentials**: Generates an htpasswd file using `apache2-utils` (with a Python fallback) for the management proxy's basic auth. Updates existing file if already present.
- **Initial nginx config**: Writes `$PROJECT_DIR/config/nginx/config.json` with the management domain and auth settings pre-configured. Skips if the file already exists (preserves user-customized config).
- **Initial firewall config**: Writes `$PROJECT_DIR/config/firewall/config.json` with auto-detected WAN/LAN interfaces. Skips if the file already exists.
- **Systemd units**: Installs three units (rendered from Jinja2 templates):
- `vacuum-wall.service` — the Flask WebUI backend.
- `vacuum-wall-acme.service` — the certificate renewal oneshot.
@@ -96,9 +98,21 @@ The installer performs the following steps automatically:
- **Firewalld zones**: Creates initial zones:
- `internal` — trusted LAN zone with DHCP, DNS, and NTP services allowed.
- `vpn` — WireGuard tunnel zone.
- **Service startup**: Enables and starts nginx, the WebUI service, and the ACME renewal timer.
- **Service startup**: Enables and starts/restarts nginx and the WebUI service, and enables the ACME renewal timer. nginx is reloaded (or restarted) to pick up any config changes.
- **ACME registration**: Registers the ACME account with the provided email via acme.sh.
### Idempotent Re-Runs
`install.sh` is fully idempotent and safe to run multiple times. Re-running the script:
- Rebuilds the Python venv and reinstalls dependencies
- Restarts `vacuum-wall` and reloads `nginx` to pick up changes
- Preserves existing SSL certificates (skips self-signed generation if a cert exists)
- Preserves existing `config.json` files (skips initial write if file exists)
- Safely updates `htpasswd` (uses update mode instead of create mode)
This makes it safe for development workflows: simply run `bash install.sh` again to update an existing installation.
---
## Post-Installation
@@ -240,7 +254,7 @@ journalctl -u nginx --no-pager -n 50
nginx -t
```
Common causes include port conflicts (another service on port 80/443), missing dependencies, or file permission issues on `$PROJECT_DIR/data/`.
Common causes include port conflicts (another service on port 80/443), missing dependencies, or file permission issues on `data/`.
### Firewall Rules Not Applying
@@ -277,7 +291,7 @@ Verify that:
- The LAN interface is assigned to a firewalld zone (check the **Interfaces** tab or `firewall-cmd --get-active-zones`).
- Dnsmasq is running: `systemctl status dnsmasq`.
- A DHCP range is configured for the correct interface. Check dnsmasq config at `$PROJECT_DIR/data/dnsmasq/`.
- A DHCP range is configured for the correct interface. Check dnsmasq config at `data/dnsmasq/`.
- The firewall allows DHCP traffic on the internal zone: `firewall-cmd --zone=internal --list-services` should include `dhcp` and `dns`.
### WebUI Not Accessible
@@ -294,10 +308,10 @@ Verify that:
| Component | Service | Config Location |
|---|---|---|
| WebUI backend | `vacuum-wall.service` | `$PROJECT_DIR/webui/` |
| WebUI backend | `vacuum-wall.service` | `webui/` |
| Reverse proxy | `nginx` | `/etc/nginx/conf.d/vacuum-wall-mgmt.conf` |
| Firewall | `firewalld` | Managed via WebUI and `firewall-cmd` |
| DHCP/DNS | `dnsmasq` | `$PROJECT_DIR/config/dnsmasq/` |
| VPN | wireguard-tools | `$PROJECT_DIR/config/wireguard/` |
| DHCP/DNS | `dnsmasq` | `config/dnsmasq/` |
| VPN | wireguard-tools | `config/wireguard/` |
| Certificates | `vacuum-wall-acme.timer` | `~/.acme.sh/` |
| Sudoers | — | `/etc/sudoers.d/vacuum-wall` |
+11 -5
View File
@@ -29,7 +29,7 @@ WireGuard support provides server-side VPN tunnel management. Peers are added th
## Tech Stack
- Debian 13 (trixie) target platform
- Python 3, Flask 3.x for web management
- Python 3.13+, Flask 3.x for web management
- firewalld (nftables backend)
- nginx 1.26+
- dnsmasq
@@ -40,14 +40,17 @@ WireGuard support provides server-side VPN tunnel management. Peers are added th
## Quick Start
To install Vacuum Wall on a Debian 13 system, run `install.sh` as root with the required environment variables:
To install Vacuum Wall on a Debian 13 system, run `install.sh` as root with required settings (CLI flags or environment variables):
```bash
MGMT_PASS=yourpassword ACME_EMAIL=admin@example.com \
bash install.sh
# Production
./install.sh --mgmt-pass yourpassword --acme-email "admin@example.com"
# Development (auto-detects your user)
./install.sh --dev --mgmt-pass yourpassword --acme-email "admin@example.com"
```
After installation, access the management interface at `https://<hostname>.local` using the credentials you configured. The `install.sh` script auto-detects the system hostname (use `MGMT_DOMAIN` to override), provisions nginx, sets up authentication, generates an initial self-signed certificate, and starts all services.
After installation, access the management interface at `https://<hostname>.local` using the credentials you configured. The `install.sh` script auto-detects the system hostname, network interfaces, and provisions nginx, authentication, an initial self-signed certificate, and all services. Run `./install.sh --help` for all options.
## Project Structure
@@ -74,6 +77,8 @@ After installation, access the management interface at `https://<hostname>.local
│ ├── dnsmasq.conf # Dnsmasq template (rendered at runtime)
│ └── wireguard*.conf # WireGuard templates (rendered at runtime)
├── lib/ # Subsystem abstraction layer
│ ├── common.py # Shared utilities (run, run_proc, load_json, save_json, deep_merge, ensure_dirs)
│ ├── logging.py # Logging setup
│ ├── firewall.py # firewalld bindings
│ ├── dnsmasq.py # DHCP/DNS configuration
│ ├── nginx.py # Reverse proxy configuration
@@ -82,6 +87,7 @@ After installation, access the management interface at `https://<hostname>.local
├── webui/ # Flask web application
│ ├── server.py # Application entry point
│ ├── api/ # REST API route modules
│ │ └── common.py # Shared API response helpers (_ok, _error)
│ ├── templates/ # Jinja2/HTMX templates
│ └── static/ # CSS and client-side JS
└── docs/ # Documentation
+3 -4
View File
@@ -39,7 +39,7 @@ Key safety properties:
### Management Interface
The Flask WebUI binds exclusively to `127.0.0.1:9090`. It is not exposed directly to any network interface. All external access to the management UI is routed through an nginx reverse proxy on the designated management domain, which provides SSL termination and HTTP Basic Authentication. The `.htpasswd` file is stored at `$PROJECT_DIR/data/nginx/.htpasswd`.
The Flask WebUI binds exclusively to `127.0.0.1:9090`. It is not exposed directly to any network interface. All external access to the management UI is routed through an nginx reverse proxy on the designated management domain, which provides SSL termination and HTTP Basic Authentication. The `.htpasswd` file is stored at `data/nginx/.htpasswd`.
### Proxy Domains
@@ -71,8 +71,7 @@ The `vacuum-wall.service` unit file applies a comprehensive set of systemd sandb
| Directive | Value | Effect |
|---|---|---|
| `ProtectSystem` | `strict` | Mounts the entire file system as read-only, except explicitly allowed paths |
| `ProtectHome` | `read-only` | Makes `/home`, `/root`, and `/run/user` inaccessible |
| `ReadWritePaths` | `$PROJECT_DIR/config`, `$PROJECT_DIR/data`, and `/tmp` | Both the application config directory and data directory, plus `/tmp`, are writable. The project path is templated at install time. |
| `ReadWritePaths` | `$INSTALL_DIR`, `$INSTALL_DIR/config`, `$INSTALL_DIR/data`, and `/tmp` | The project directory, config directory, data directory, and `/tmp` are writable (required by `ProtectSystem=strict`). The project path is templated at install time. |
| `PrivateTmp` | `yes` | Provides a private `/tmp` and `/var/tmp` namespace |
| `NoNewPrivileges` | `yes` | Prevents the process from gaining new privileges via `setuid`/`setgid` |
| `IPAddressDeny` | `all` | Drops all network traffic |
@@ -89,7 +88,7 @@ The `vacuum-wall.service` unit file applies a comprehensive set of systemd sandb
| `SystemCallFilter` | `@system-service` | Allows only a curated set of system calls safe for services |
| `RestrictRealtime` | `yes` | Prevents the process from acquiring realtime scheduling priorities |
The `User`, `Group`, `WorkingDirectory`, `ExecStart`, and `ReadWritePaths` directives in the service unit are rendered from a Jinja2 template at install time with the configured `USER_NAME` and `PROJECT_DIR`.
The `User`, `Group`, `WorkingDirectory`, `ExecStart`, and `ReadWritePaths` directives in the service unit are rendered from a Jinja2 template at install time with the configured `USER_NAME` and `INSTALL_DIR`.
This hardening ensures that even if the Flask application is compromised, the attacker is confined to a sandboxed environment with no direct network access, no write access outside the config and data directories, and no ability to escalate privileges through kernel interfaces.