microsoft / microsoft/winml-cli

[eval] DRY: drop repeated "winml.modelkit.eval." prefix in _EVALUATOR_REGISTRY

Open Beginner friendly
#808 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

P2 refactor triaged
Dominant language
Python
Stars
40
Forks
11
Avg merge
1d 8h
Merged PRs (30d)
50

Description

Component: eval
Type: cleanup / quality

Context

eval/evaluate.py::_EVALUATOR_REGISTRY maps task -> "module_path:ClassName", e.g.:

"image-feature-extraction":
    "winml.modelkit.eval.image_feature_extraction_evaluator:WinMLImageFeatureExtractionEvaluator",

The winml.modelkit.eval. prefix is repeated for all 15 entries. Because the full single-line entries exceed the 100-char limit, the dict is currently wrapped in # fmt: off / # fmt: on to preserve the readable key/value-per-line layout (otherwise ruff-format collapses each entry onto one >100-char line, which trips E501). Introduced/surfaced in #807.

Proposed cleanup

  • Store the relative "<module>:<ClassName>" (drop the repeated package prefix).
  • Prepend winml.modelkit.eval. once in get_evaluator_class: importlib.import_module(f"winml.modelkit.eval.{module_path}").
  • Most entries then fit on a single line naturally; only the 3 longest (image-feature-extraction, zero-shot-classification, zero-shot-image-classification) still need # noqa: E501.
  • This removes the # fmt: off / # fmt: on block.

Note

tests/unit/eval/test_eval.py::test_registered_task_returns_class asserts cls.__module__ == module_path; update it to compare against the full prefixed path.

Out of scope

Behavior is unchanged — purely a storage-format/readability cleanup.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in eval/evaluate.py at _EVALUATOR_REGISTRY and get_evaluator_class, then read tests/unit/eval/test_eval.py::test_registered_task_returns_class. Store registry entries without the repeated package prefix, add the single prefix at import time, update the module-path assertion, and run the focused test to confirm behavior is unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Refactor
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.