microsoft / microsoft/winml-cli

depth_anything + dpt + zoedepth / depth-estimation: all models pass wmk perf

Open
#123 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

model / task scale P2 triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Summary

All depth estimation models fail. depth_anything and zoedepth fail because no OnnxConfig is registered for their model types. dpt models fail due to an import error (build_hf_model not found) and an incomplete export pipeline.

Eval Results (2026-03-11)

Status Model model_type Error
FAIL depth-anything/Depth-Anything-V2-Small-hf depth_anything No OnnxConfig registered for model_type='depth_anything'
FAIL depth-anything/Depth-Anything-V2-Base-hf depth_anything same
FAIL depth-anything/Depth-Anything-V2-Large-hf depth_anything same
FAIL LiheYoung/depth-anything-small-hf depth_anything same
FAIL LiheYoung/depth-anything-base-hf depth_anything same
FAIL LiheYoung/depth-anything-large-hf depth_anything same
FAIL Intel/dpt-large dpt cannot import name 'build_hf_model' from 'modelkit.build'
FAIL Intel/dpt-hybrid-midas dpt Export pipeline stalls
FAIL Intel/zoedepth-nyu-kitti zoedepth No OnnxConfig registered for model_type='zoedepth'
FAIL apple/DepthPro-hf depth_pro No OnnxConfig registered for model_type='depth_pro'

10/10 models fail — 0 pass.

Root Cause

  1. depth_anything, zoedepth, depth_pro: ModelKit's modelkit/models/hf/depth_anything.py exists but imports only trigger registration of DepthAnythingIOConfig. Check whether the ONNX config is correctly registered for all sub-variants (depth_anything, depth_anything_v2, zoedepth, depth_pro).

  2. dpt: cannot import name 'build_hf_model' from 'modelkit.build' indicates a stale import path — likely a refactor that moved build_hf_model without updating all import sites. Also has export pipeline stall for dpt-hybrid-midas.

Current State

  • modelkit/models/hf/depth_anything.py — exists, registers DepthAnythingIOConfig, but may not cover all depth model types
  • No dpt.py or zoedepth.py in modelkit/models/hf/
  • modelkit/build/__init__.pybuild_hf_model import broken for dpt

Desired State

All 10 depth estimation models above pass wmk perf.

Acceptance Criteria

  • All 6 depth_anything (v1 + v2) models pass wmk perf
  • Both dpt models pass wmk perf
  • Intel/zoedepth-nyu-kitti passes wmk perf
  • apple/DepthPro-hf passes wmk perf
  • Fix is universal — no hardcoded model variant names (CLAUDE.md Cardinal Rule #1)
  • uv run pytest tests/ passes (CLAUDE.md Cardinal Rule #3)
  • Eval re-run confirms 10/10 pass

Technical Notes

  • Check modelkit/models/hf/depth_anything.py — verify it registers configs for depth_anything, depth_anything_v2, and related model_type strings
  • For zoedepth and depth_pro: add ONNX config registrations (they are ViT-based depth models similar to DPT/DepthAnything)
  • For dpt import error: trace the build_hf_model import chain in modelkit/build/__init__.py and fix the broken import — likely a refactor artifact
  • DPTForDepthEstimation uses a ViT backbone — check whether the ViT ONNX config is compatible

Related Files

  • modelkit/models/hf/depth_anything.py — existing depth_anything config (needs audit)
  • modelkit/build/__init__.py — broken build_hf_model import for dpt
  • modelkit/models/hf/__init__.py:32depth_anything import line
  • eval_results/2026-03-11/models/Intel__dpt-large__depth-estimation/result.json
  • eval_results/2026-03-11/models/depth-anything__Depth-Anything-V2-Small-hf__depth-estimation/result.json

References

  • CLAUDE.md Cardinal Rule #1: No hardcoded model type strings
  • CLAUDE.md Cardinal Rule #3: Run uv run pytest tests/ after implementation

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with modelkit/models/hf/depth_anything.py, modelkit/models/hf/init.py:32, and modelkit/build/init.py; trace the reported registration and build_hf_model failures, then inspect the referenced eval result files. Done means all 10 listed models pass wmk perf and uv run pytest tests/ passes without hardcoded model variants.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.