pyronear / pyronear/pyro-annotator

feat(api): validate sequence annotation completeness on transition to seq_annotation_done

Open
#151 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
0
Forks
1
Avg merge
1d 2h
Merged PRs (30d)
2

Description

Context

The invariant "every object in a sequence annotation must be classified before the sequence reaches seq_annotation_done" is currently enforced only in the frontend:

  • frontend/src/utils/annotation/progressUtils.tsisAnnotationComplete() requires bboxes.every(hasUserAnnotations) plus a missed-smoke review.
  • frontend/src/utils/annotation/sequenceUtils.tshasUserAnnotations(): a bbox is classified iff (is_smoke=true and smoke_type set) or false_positive_types is non-empty.
  • frontend/src/components/sequence-annotation/AnnotationHeader.tsx — save button disabled unless complete or marked unsure.

The API accepts any processing_stage with any payload — nothing re-validates completeness server-side.

Why it matters

The frontend is not the only writer: push_sequence_annotations.py, update_annotation_stage.py (make update-stage-remote), import.py, and any raw API client can move an annotation into seq_annotation_done.

The detection-review pipeline (B) then trusts the stage blindly and silently mislabels incomplete data: collect_annotation_bboxes() in pull_sequence_annotations.py treats an object with is_smoke=false and empty false_positive_types — i.e. an unreviewed object — as a false positive and exports it as fp_unlabeled. If the FiftyOne reviewer doesn't catch it, apply_fiftyone_review.py deletes the remote detection annotations and recreates them with that wrong label at stage annotated. This is the same class of classification-fidelity bug as #148/#150.

Proposal

When a sequence annotation is created or updated with processing_stage = seq_annotation_done and is_unsure = false, validate that every entry in annotation.sequences_bbox satisfies:

  • is_smoke = true and smoke_type is set, or
  • false_positive_types is non-empty.

Return 422 with a field-level error otherwise. Natural home: the existing validation layer in app/schemas/annotation_validation.py / the sequence-annotation endpoints (which already derive has_smoke / has_false_positives server-side, so there is precedent for annotation-content logic in the API).

Caveats

  1. Validate on the transition (POST/PATCH carrying that stage). A stage-only PATCH must validate the stored annotation payload.
  2. Audit existing rows first: any legacy incomplete annotation would make stage-only updates start failing (update_annotation_stage.py, the push script). Probably desirable failures, but the count should be known before enabling.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with app/schemas/annotation_validation.py and the sequence-annotation endpoints, then inspect how POST/PATCH requests derive has_smoke and has_false_positives. Audit existing rows before enabling the check, including stage-only updates that use the stored payload. Done means transitions to seq_annotation_done with is_unsure=false reject incomplete sequence objects with a 422 field-level error, while complete payloads continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, typescript
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.