pyronear / pyronear/pyro-annotator
feat(api): validate sequence annotation completeness on transition to seq_annotation_done
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 1
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 2
Description
Context
The invariant "every object in a sequence annotation must be classified before the sequence reaches seq_annotation_done" is currently enforced only in the frontend:
frontend/src/utils/annotation/progressUtils.ts—isAnnotationComplete()requiresbboxes.every(hasUserAnnotations)plus a missed-smoke review.frontend/src/utils/annotation/sequenceUtils.ts—hasUserAnnotations(): a bbox is classified iff (is_smoke=trueandsmoke_typeset) orfalse_positive_typesis non-empty.frontend/src/components/sequence-annotation/AnnotationHeader.tsx— save button disabled unless complete or marked unsure.
The API accepts any processing_stage with any payload — nothing re-validates completeness server-side.
Why it matters
The frontend is not the only writer: push_sequence_annotations.py, update_annotation_stage.py (make update-stage-remote), import.py, and any raw API client can move an annotation into seq_annotation_done.
The detection-review pipeline (B) then trusts the stage blindly and silently mislabels incomplete data: collect_annotation_bboxes() in pull_sequence_annotations.py treats an object with is_smoke=false and empty false_positive_types — i.e. an unreviewed object — as a false positive and exports it as fp_unlabeled. If the FiftyOne reviewer doesn't catch it, apply_fiftyone_review.py deletes the remote detection annotations and recreates them with that wrong label at stage annotated. This is the same class of classification-fidelity bug as #148/#150.
Proposal
When a sequence annotation is created or updated with processing_stage = seq_annotation_done and is_unsure = false, validate that every entry in annotation.sequences_bbox satisfies:
is_smoke = trueandsmoke_typeis set, orfalse_positive_typesis non-empty.
Return 422 with a field-level error otherwise. Natural home: the existing validation layer in app/schemas/annotation_validation.py / the sequence-annotation endpoints (which already derive has_smoke / has_false_positives server-side, so there is precedent for annotation-content logic in the API).
Caveats
- Validate on the transition (POST/PATCH carrying that stage). A stage-only PATCH must validate the stored annotation payload.
- Audit existing rows first: any legacy incomplete annotation would make stage-only updates start failing (
update_annotation_stage.py, the push script). Probably desirable failures, but the count should be known before enabling.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with app/schemas/annotation_validation.py and the sequence-annotation endpoints, then inspect how POST/PATCH requests derive has_smoke and has_false_positives. Audit existing rows before enabling the check, including stage-only updates that use the stored payload. Done means transitions to seq_annotation_done with is_unsure=false reject incomplete sequence objects with a 422 field-level error, while complete payloads continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, typescript
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100