ByteDance-Seed / ByteDance-Seed/Depth-Anything-3

Streaming/Real-time, MPS, Benchmarking - PR incoming

Open
#49 1 comment 4 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.3k
Forks
702
PR merge metrics
No merged PRs in 30d

Description

In the process of preparing a PR here for all this. Until then, i'll just leave this here for anyone looking for pointers for any of this.

PR in my fork:
https://github.com/laubsauger/Depth-Anything-3/pull/1

Image

## Performance & Infrastructure

**Real-time Streaming Architecture**
- WebSocket streaming server for live depth estimation
- Sliding window processing for infinite-length video streams
- Device-specific buffer defaults (12 frames MPS, 24 CUDA)
- Configurable latency/quality tradeoffs via FPS limits and quality settings

**macOS Apple Silicon Support**
- MPS backend support throughout codebase
- Installation script for automatic setup
- gsplat-mps integration for 3D Gaussian Splatting on Apple GPUs

**Benchmarking System**
- Automated performance testing across all models (DA3-SMALL to DA3NESTED-GIANT-LARGE)
- Multi-device benchmarking (MPS, CUDA, CPU)
- Resolution scaling tests (252px to 1008px)
- Batch size performance profiling
- Memory usage tracking and optimization analysis
- Statistical metrics (mean, median, std dev) across multiple runs

**Network & Protocol Optimizations**
- Raw float32 numpy protocol for lossless streaming
- JPEG compression with quality controls for bandwidth reduction
- Sequence ID tagging for depth/rgb frame synchronization

## TouchDesigner Integration (Optional)

**Complete Component System**
- Centralized parameter management via extension class
- Frame synchronization using sequence IDs and circular buffers
- Three synchronized outputs: latest input, depth map, and matched RGB
- Aspect-aware resolution scaling for GPU downsampling
- Compatible with vvvv, Processing, Max/MSP via same WebSocket protocol

---

Image

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.