bmaltais / bmaltais/gaussify

gaussify: CLI tool for end-to-end Gaussian splatting from video

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Problem Statement & Solution

Technical hobbyists who want to create Gaussian splats from video footage face a fragmented, error-prone setup process. Creating a splat currently requires manually downloading and extracting multiple native binaries (ffmpeg, COLMAP, GLOMAP, Brush), editing Windows PATH environment variables, running a Python script with specific arguments, and debugging cryptic tool failures with no guidance on which step broke or why. Each tool has its own installation quirks, and the combined failure surface is large enough that most users give up before producing their first splat.

\`gaussify\` is a cross-platform CLI tool that owns the full pipeline: one install command bootstraps all required binaries into a local project directory, and one run command takes a video file to a \`.splat\` file on disk. The tool detects the available GPU backend (CUDA, ROCm, or CPU) and selects the appropriate binaries automatically. When a step fails, the CLI identifies which stage broke and surfaces the tool's raw output alongside a clear stage label — no silent failures, no guessing.

The solution deliberately avoids global system changes. All binaries are isolated within the project directory, nothing is added to PATH, and the Python environment is managed with \`uv\`. This keeps the tool reproducible, easy to remove, and safe to run alongside other tools without conflict.

## User Stories

1. As a **hobbyist filmmaker**, I want to run a single install command that downloads all required tools so that I don't have to manually track down binaries, extract archives, or edit environment variables.

2. As a **hobbyist filmmaker**, I want to convert a video file to a \`.splat\` file with one command so that I can focus on capturing footage rather than learning four different CLI tools.

3. As a **Windows user with an NVIDIA GPU**, I want the tool to automatically detect my CUDA installation and select CUDA-compatible binaries so that I get maximum performance without manually specifying GPU flags.

4. As a **Linux user with an AMD GPU**, I want ROCm-compatible binaries to be selected automatically so that I'm not forced to use CPU-only processing.

5. As a **user whose pipeline run fails**, I want the CLI to clearly report which stage failed (frame extraction, SfM, or 3DGS training) and print the tool's raw output so that I can diagnose the problem without guessing.

6. As a **user running the tool for the first time**, I want the CLI to detect missing tools and prompt me to run the install command so that I get a clear recovery path rather than a cryptic binary-not-found error.

7. As a **user working on multiple scenes**, I want each project directory to contain its own isolated copy of all tools so that updating one project's tools doesn't break another.

8. As a **hobbyist with limited VRAM**, I want the frame count to be configurable so that I can trade reconstruction quality for faster processing and lower memory usage.

## Implementation Decisions

**Pipeline stages:** The pipeline is fixed and sequential — frame extraction (ffmpeg) → Structure from Motion (COLMAP + GLOMAP) → 3D Gaussian Splatting training (Brush) → output file. Each stage must complete successfully before the next begins. This matches the established photogrammetry workflow and gives clear failure attribution.

**Frame extraction via ffmpeg:** Blender is excluded from the pipeline. ffmpeg handles frame extraction because it is CLI-native, scriptable, and has a mature cross-platform binary distribution. This eliminates a heavyweight GUI dependency and keeps the tool fully headless.

**3DGS engine — Brush:** Brush is chosen over alternatives because it is actively maintained, has cross-platform GPU support including experimental Metal on macOS, and provides a clean CLI interface suitable for pipeline integration.

**Binary isolation strategy:** All native binaries (ffmpeg, COLMAP, GLOMAP, Brush) are downloaded from their official GitHub Releases into a \`.tools/\` directory inside the project. No global PATH modifications are made. The Python orchestrator, managed by \`uv\`, is responsible for locating and invoking these binaries using their absolute local paths. This approach was chosen over a package manager (e.g. pixi/conda) to avoid adding a non-Python bootstrap dependency.

**GPU auto-detection:** At runtime, the CLI detects the available GPU compute backend (NVIDIA CUDA, AMD ROCm, or CPU fallback) and selects the matching pre-built binary for Brush. The user can override this with an explicit \`--gpu\` flag. Detection failure defaults to CPU with a warning.

**Error handling:** When any stage fails, the CLI prints the stage name, the exact command that was run, and the tool's raw stdout/stderr output. No retry logic, no automatic parameter adjustment. The goal is transparency, not magic recovery.

**Python toolchain:** The orchestrator is a \`uv\`-managed Python package. \`uv\` is the only bootstrap requirement. This gives reproducible environments, fast installs, and no interference with system Python.

## Testing Decisions

- **Unit tests** must cover GPU detection logic across all three backend cases (CUDA present, ROCm present, neither) using environment stubs — no real GPU required in CI.
- **Unit tests** must cover the binary downloader for each supported platform (Windows x64, Linux x64, macOS arm64) by mocking HTTP responses and verifying the correct release asset URL is constructed and the binary lands in the expected local path.
- **Integration tests** must run the full pipeline against a small synthetic image set (pre-generated frames, skipping ffmpeg) and verify that COLMAP + GLOMAP produce a valid sparse reconstruction and Brush produces a non-empty output file. These tests require a machine with an actual GPU and must be tagged to run only in GPU CI environments.
- **CLI contract tests** must verify that \`gaussify install\` and \`gaussify run\` exit with non-zero codes and print the failing stage name when a tool binary is missing or returns a non-zero exit code.
- All tests must be runnable offline after initial binary download. No test should require internet access at assertion time.

## Out of Scope & Further Notes

**Out of scope (Phase 1):**
- GUI — deferred to Phase 2
- Viewer integration — users open the output \`.splat\` in their viewer of choice
- macOS Metal support — Brush Metal is experimental; detection is included but not tested
- Photo-only input (images, no video) — deferred; the current pipeline assumes a video source
- Cloud/remote processing — local GPU execution only
- Blender photogrammetry importer integration — removed from pipeline entirely
- Automatic CUDA/ROCm driver installation — system-level dependency, out of scope by design
- Version upgrade command (\`gaussify upgrade\`) — deferred

**Further notes:**
- The \`.tools/\` directory should be added to \`.gitignore\` by the install command automatically.
- GLOMAP is the accelerated SfM backend; COLMAP is still required as it handles feature extraction and matching that GLOMAP delegates to it. Both must be installed.
- A \`gaussify doctor\` command (checks tool presence, GPU detection, reports versions) would be high value as a Phase 1 follow-on, but is not required for the initial release.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no existing files, tests, or entry points; start by inspecting the repository and locating the Python/uv package and CLI entry points. Review the required GPU, downloader, pipeline, and CLI contract test cases, then confirm that the Phase 1 pipeline, supported platforms, and offline-test constraints are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, computer-vision
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.