microsoft / microsoft/winml-cli
[eval] DRY: drop repeated "winml.modelkit.eval." prefix in _EVALUATOR_REGISTRY
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 40
- Forks
- 11
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 50
Description
Component: eval
Type: cleanup / quality
Context
eval/evaluate.py::_EVALUATOR_REGISTRY maps task -> "module_path:ClassName", e.g.:
"image-feature-extraction":
"winml.modelkit.eval.image_feature_extraction_evaluator:WinMLImageFeatureExtractionEvaluator",
The winml.modelkit.eval. prefix is repeated for all 15 entries. Because the full single-line entries exceed the 100-char limit, the dict is currently wrapped in # fmt: off / # fmt: on to preserve the readable key/value-per-line layout (otherwise ruff-format collapses each entry onto one >100-char line, which trips E501). Introduced/surfaced in #807.
Proposed cleanup
- Store the relative
"<module>:<ClassName>"(drop the repeated package prefix). - Prepend
winml.modelkit.eval.once inget_evaluator_class:importlib.import_module(f"winml.modelkit.eval.{module_path}"). - Most entries then fit on a single line naturally; only the 3 longest (
image-feature-extraction,zero-shot-classification,zero-shot-image-classification) still need# noqa: E501. - This removes the
# fmt: off/# fmt: onblock.
Note
tests/unit/eval/test_eval.py::test_registered_task_returns_class asserts cls.__module__ == module_path; update it to compare against the full prefixed path.
Out of scope
Behavior is unchanged — purely a storage-format/readability cleanup.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in eval/evaluate.py at _EVALUATOR_REGISTRY and get_evaluator_class, then read tests/unit/eval/test_eval.py::test_registered_task_returns_class. Store registry entries without the repeated package prefix, add the single prefix at import time, update the module-path assertion, and run the focused test to confirm behavior is unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100