ByteDance-Seed / ByteDance-Seed/Depth-Anything-3

Feature Request: Add Hugging Face Transformers Pipeline Support for DA3 Models

Open
#182 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
6.3k
Forks
702
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature Request

### Is your feature request related to a problem? Please describe.

Currently, Depth Anything V3 models require custom installation and usage through the `depth_anything_3` package. While this works well, it creates friction for users who are already using the Hugging Face Transformers ecosystem and would like to seamlessly integrate DA3 models into their existing workflows.

Unlike Depth Anything V2 models (which work via `transformers.pipeline`), DA3 models are not yet integrated into the Transformers library, requiring users to maintain separate codebases and dependencies.

**Example of current limitation:**

```python
# This works for V2 ✅
from transformers import pipeline
pipe = pipeline(task='depth-estimation', model='depth-anything/Depth-Anything-V2-Small-hf')
depth = pipe(image)

# This fails for V3 ❌
pipe = pipeline(task='depth-estimation', model='depth-anything/DA3-BASE')
# ValueError: Unrecognized model in depth-anything/DA3-BASE
```

### Describe the solution you'd like

Add official support for DA3 models in the Hugging Face Transformers library, enabling users to:

1. **Use the standard `pipeline` API** for depth estimation with DA3 models
2. **Seamlessly switch** between V2 and V3 models without changing code structure
3. **Leverage existing Transformers features** like automatic model caching, device mapping, and preprocessing

**Desired API:**

```python
from transformers import pipeline

# Should work seamlessly
pipe = pipeline(
task='depth-estimation',
model='depth-anything/DA3-BASE',
device='mps' # or 'cuda', 'cpu'
)

result = pipe(image)
depth_map = result['depth']
```

### Describe alternatives you've considered

**Current workaround:**

```python
from depth_anything_3.api import DepthAnything3
model = DepthAnything3.from_pretrained('depth-anything/DA3-BASE')
prediction = model.inference([image])
```

While this works, it requires:
- Separate installation (`pip install -e .` from GitHub)
- Different API/code patterns
- Manual device management
- Breaking changes when migrating from V2 to V3

### Benefits

Adding Transformers support would:

1. ✅ **Improve adoption** - Lower barrier to entry for existing Transformers users
2. ✅ **Standardize usage** - Consistent API across depth estimation models
3. ✅ **Enable comparison** - Easy A/B testing between V2 and V3
4. ✅ **Leverage ecosystem** - Automatic benefits from Transformers improvements
5. ✅ **Simplify deployment** - Standard model serving patterns (e.g., inference endpoints)

### Additional context

- **Models are already on Hugging Face Hub**: https://huggingface.co/depth-anything/DA3-BASE
- **License is Apache 2.0**: Compatible with commercial use
- **V2 integration exists**: Proven pattern for depth estimation in Transformers
- **Use case**: Building production applications that need to support both V2 and V3 with minimal code changes

### Environment

- OS: macOS (Apple Silicon M3)
- Python: 3.13
- Transformers: 4.x
- Use case: Interior design visualization pipeline

### Would you be willing to contribute?

I'd be happy to help test the integration or provide feedback during development.

### References

- Depth Anything V2 (working example): https://huggingface.co/depth-anything/Depth-Anything-V2-Small-hf
- DA3 Model Hub: https://huggingface.co/depth-anything/DA3-BASE
- Transformers Pipeline Docs: https://huggingface.co/docs/transformers/main_classes/pipelines

---

Thank you for developing Depth Anything V3! The improvements over V2 are impressive, and Transformers integration would make it even more accessible to the community. 🙏

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the existing Depth Anything V2 Transformers pipeline integration with the depth_anything_3.api.DepthAnything3 usage and the referenced DA3-BASE model. Identify the repository entry points and tests needed for pipeline('depth-estimation', ...) support, then verify the documented device options and that the result exposes a 'depth' value.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.