ml-explore / ml-explore/mlx-examples

Community project: Depth-Anything-V2 running via a from-scratch PyTorch-API shim (torch-mlx)

Open
#1,449 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
9k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

Hi — sharing a community project in case it's of interest, and to ask whether you'd consider a pointer to it somewhere (this repo, a discussion, or wherever you think fits best).

depth-anything-mlx: runs the real, unmodified transformers.AutoModelForDepthEstimation (Depth-Anything-V2) on Apple Silicon via torch-mlx — a from-scratch torch-API-compatible layer backed by mlx.core — instead of a hand-written native mlx.nn reimplementation.

I want to be upfront that this takes a different approach from every example in this repo: mlx-examples' convention (e.g. clip/, segment_anything/) is a native mlx.nn model definition, with torch used only for one-time offline weight conversion. depth-anything-mlx instead runs the real PyTorch model through a compatibility shim at inference time, every call — so it isn't a fit for an inline PR into the example gallery itself, which is why I'm opening this as an issue instead rather than a pull request.

Despite the different approach, the results seemed worth sharing:

  • Fully verified against real PyTorch at every stage (weights, preprocessing, model forward) — see the repo's BENCHMARK_RESULTS.md for the full accounting, including negative results.
  • After a series of fixes (fused attention/layer-norm kernels, fp16, native preprocessing, an exact conv-fold), it now beats real PyTorch's own MPS backend by a consistent ~1.4-1.5x across image sizes from 480x640 up to 108MP.

Happy to answer any questions, and no worries at all if this isn't a fit for anything here — just wanted to put it in front of the people most likely to find it interesting.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HooZphcpzukceMrdArswsP

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository file or test is named. Start by reviewing the linked depth-anything-mlx and torch-mlx projects, including BENCHMARK_RESULTS.md, then determine whether and where a project pointer belongs; done would be a clearly agreed documentation or discussion location.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.