pyronear / pyronear/temporal-model

eval: evaluate_pyro_annotator omits transitive core deps (tubes.py, types.py, ...)

Open Beginner friendly
#66 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1
Forks
0
Avg merge
12h 43m
Merged PRs (30d)
7

Description

evaluate_pyro_annotator in eval/dvc.yaml declares fewer ../core deps than evaluate@train/@val, but reaches the missing ones transitively at runtime: inference.py imports .tubes/.types/.logistic_calibrator, and model.py imports .package/.tubes/.types. Missing from the dep list: tubes.py, types.py, logistic_calibrator.py, sequences.py, package.py, temporal_classifier.py.

Consequence: a change that touches only e.g. core/tubes.py (tube linking — exactly what the pyro-annotator scoring exercises) leaves the stage's lock untouched and dvc repro declares it up to date, so the stage the retrain runbook designates as the acceptance gate can report metrics produced by different code than reviewed. The v4.1.0 run wasn't affected only because model.zip changed too.

Fix: align the dep list with evaluate@train/@val (or factor the shared list). Noticed in the PR #65 review; kept out of that PR to avoid scope creep.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in eval/dvc.yaml by comparing evaluate_pyro_annotator with evaluate@train and @val. Check the dependency entries for the six named core files, then verify that a change to one of them causes the evaluation stage to be considered out of date by DVC. Done means the pyro-annotator stage tracks the same relevant dependencies as the other evaluation stages.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.