86 lines
3.9 KiB
Markdown
86 lines
3.9 KiB
Markdown
# AGENTS.md
|
|
|
|
## Setup (order matters)
|
|
|
|
```bash
|
|
# 1. Clone with submodule (libs/LTX-2 is a git submodule, mandatory)
|
|
git clone --recursive <repo> revids && cd revids
|
|
|
|
# 2. Create venv and install LTX packages from submodule in editable mode
|
|
python -m venv .venv && source .venv/bin/activate
|
|
pip install -e libs/LTX-2/packages/ltx-core -e libs/LTX-2/packages/ltx-pipelines
|
|
|
|
# 3. Install API dependencies
|
|
pip install -r requirements.txt
|
|
|
|
# 4. Download ~30 GB of model weights (required before first run)
|
|
huggingface-cli download Lightricks/LTX-2.3 --include "ltx-2.3-22b-distilled-1.1.safetensors" --local-dir models/
|
|
huggingface-cli download Lightricks/LTX-2.3 --include "ltx-2.3-spatial-upscaler-x2-1.1.safetensors" --local-dir models/
|
|
huggingface-cli download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b-it-qat-q4_0-unquantized
|
|
|
|
# 5. Set env (must be set before every run to avoid GPU OOM)
|
|
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
|
|
```
|
|
|
|
## Run the server
|
|
|
|
```bash
|
|
# Development (auto-reload, sets CUDA alloc conf)
|
|
./run-dev.sh
|
|
|
|
# Production (no reload)
|
|
./run.sh
|
|
|
|
# Manual (equivalent)
|
|
.venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
|
```
|
|
|
|
API is at `http://localhost:8000`, Swagger docs at `/docs`.
|
|
|
|
## Configuration
|
|
|
|
All settings via `.env` file or `REVIDS_`-prefixed env vars. See `app/config.py` and `.env.example`. Key settings:
|
|
- `REVIDS_LTX_DISTILLED_CHECKPOINT` — distilled model `.safetensors` path
|
|
- `REVIDS_LTX_GEMMA_ROOT` — Gemma text encoder directory
|
|
- `REVIDS_LTX_SPATIAL_UPSAMPLER` — upsampler `.safetensors` path
|
|
- `REVIDS_LTX_QUANTIZATION` — `fp8_cast` (default, ~40% VRAM savings), `fp8_scaled_mm`, or `""` (bf16)
|
|
- `REVIDS_LTX_GEMMA_QUANTIZATION` — `nf4` for NF4-quantized Gemma (requires `bitsandbytes`), or `""`/unset for bf16
|
|
- `REVIDS_LTX_LORAS` — comma-separated `path:scale` pairs for LoRA loading
|
|
|
|
## Architecture
|
|
|
|
Single-service FastAPI app. All code lives in `app/`:
|
|
- `main.py` — FastAPI routes, HTTP handlers
|
|
- `service.py` — LTX pipeline wrapper, job lifecycle, video encoding
|
|
- `database.py` — SQLite job store (`aiosqlite`, auto-creates schema)
|
|
- `models.py` — Pydantic request/response validators
|
|
- `config.py` — `pydantic-settings` with `REVIDS_` env prefix
|
|
- `text_encoder.py` — NF4 quantized Gemma encoder; patches `ltx_pipelines` at runtime
|
|
- `cli.py` — CLI client (`revids` entry point in pyproject.toml)
|
|
|
|
### Runtime behavior
|
|
- **Pipeline loads lazily** on first job submission, not at server start
|
|
- **Single GPU lock**: jobs are serialized via `asyncio.Semaphore(1)` (configurable)
|
|
- **Jobs persist** in `app/jobs.db` (SQLite, gitignored). DB is created on first server start
|
|
- **Video output**: `videos/` directory (gitignored), files named `<job_id>.mp4`
|
|
- **Job IDs**: 12 lowercase hex chars (`uuid4().hex[:12]`)
|
|
|
|
### Pipeline patching
|
|
`service.py:_patch_ltx_pipelines()` monkeypatches `DistilledPipeline.__init__` and `ltx_pipelines.utils.blocks.PromptEncoder` at runtime to inject the project's `Nf4PromptEncoder` and wire `gemma_quantization` support. These patches run once, before the first pipeline load.
|
|
|
|
### Submodule is read-only
|
|
`libs/LTX-2/` is a read-only git submodule. Never modify files in `ltx-core` or `ltx-pipelines`. If upstream behavior needs changing, work around it in `app/` (e.g., via monkeypatching, wrappers, or config).
|
|
|
|
## Validation constraints (enforced at API layer)
|
|
- `num_frames` must be `8n+1`: 9, 17, 25…, 65, 97, 121, 161, 257
|
|
- `width` and `height` must be divisible by 32, range [256, 2048]
|
|
- `fps` must be in (0, 60]
|
|
- Uploaded image must be under 10 MB
|
|
|
|
## No tests, no lint config for the app itself
|
|
|
|
The `app/` code has no test suite, no ruff/pyright config, and no CI workflows. The `libs/LTX-2/` submodule has its own tooling (ruff, pytest) managed by `uv` — but that's separate from this project.
|
|
|
|
## Hardware
|
|
Requires NVIDIA GPU with CUDA. Minimum ~24 GB VRAM (bf16), ~14 GB with FP8 quantization.
|