pyronear / pyronear/temporal-model

Pipeline silently drops one copy when a sequence id exists in both wildfire/ and fp/ pools

Open
#68 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1
Forks
0
Avg merge
12h 43m
Merged PRs (30d)
7

Description

Found during the v4.1.0 retrain (PR #65), verified on disk. pyro-dataset v4.1.0's train split ships pyronear-sdis-07_brison_226_2023-07-14T12-40-59 under both wildfire/ and fp/ (upstream: pyronear/pyro-dataset#39). Every pipeline stage flattens sequences by bare name:

  • build_tubes writes <seq>.json into one flat dir, so the fp copy overwrote the wildfire one: _summary.json says 2968 tubes written, 2967 tube files exist, and the surviving tube is labeled fp while its patches resolve from the wildfire dir (find_sequence_dir is wildfire-first). Net: the v4.1.0 model trained with one real-smoke sequence labeled as a false positive.
  • eval has the same keyspace: results.json has 3328 rows but 3327 unique keys; details/ and sequences/ hold 3327 files.

Nothing errors: dropped.json stays empty, dvc stays green, and the per-pool counts (1664/1664) pass every sanity check — a mislabeled training sample with zero signal, recurring on any future collision.

Proposed fix: fail loudly on duplicate sequence ids across pools (train build_tubes and eval's sequence listing), or namespace outputs by label. Upstream dedup (pyronear/pyro-dataset#39) fixes v4.1.0's instance; this guard prevents the silent variant forever.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace the build_tubes and eval sequence-listing entry points, then inspect find_sequence_dir and the generated _summary.json, results.json, details/, and sequences/ artifacts. Done means duplicate sequence ids across pools can no longer silently overwrite or mismatch labels: the pipeline should fail loudly or consistently namespace outputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.