Docs proposal: review object-detection augmentation changes in pull requests with CML
- Dominant language
- TypeScript
- Stars
- 13
- Forks
- 22
- PR merge metrics
- No merged PRs in 30d
Description
# Feature request
Would you be open to a focused guide or small example repository for ML teams that need to review an object-detection augmentation-policy change in a pull request before merging it or spending compute on model training?
## User problem
A change to stochastic preprocessing can alter an image while silently mishandling its bounding boxes or class labels. A unit test can check shapes and coordinate ranges, but it does not let a reviewer see whether the transformed training samples still make sense. Discovering the problem from model metrics is late and makes the cause harder to isolate.
CML already turns generated images, tables, and Markdown into a pull-request comment or check. AlbumentationsX can apply one label-aware policy to an image, its boxes, and their class IDs, use a stable sample-level `invocation_seed`, and expose both the serialized policy and the transforms realized for a sample.
Together, they could give a reviewer a compact augmentation report directly in the pull request:
- the fixed input scene and its annotations;
- transformed samples for a small set of stable seeds;
- pass/fail annotation invariants;
- the transforms and sampled parameters realized for each preview; and
- the resolved augmentation policy used by CI.
The reviewer could inspect the data effect of the code change before approving it. The example would demonstrate a CML review workflow rather than a generic augmentation snippet.
## Proposed workflow
The example would use a generated detection scene, avoiding dataset downloads and data-license dependencies. A small Python script would:
1. construct an AlbumentationsX policy for an image, Pascal VOC bounding boxes, and `class_labels`;
2. apply it to the same fixture with stable `invocation_seed` values such as `17` and `29`;
3. assert that returned boxes and labels remain paired and that coordinates are valid for the output image;
4. render the original and transformed annotations as contact-sheet images;
5. write a compact applied-transform table from AX `run_with_trace()` output; and
6. save `A.to_dict(policy)` as JSON for policy provenance.
The GitHub Actions job would then assemble a Markdown report and publish it with the existing CML interface:
```yaml
- uses: actions/checkout@v4
- uses: iterative/setup-cml@v1
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Build augmentation review
run: |
pip install -r requirements.txt
python review_augmentation.py --output artifacts
- name: Publish CML report
env:
REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
python build_report.py --artifacts artifacts --output report.md
cml comment create report.md
```
The report could contain a small table such as:
| Input | AX policy, seed 17 | AX policy, seed 29 |
| --- | --- | --- |
| Original boxes and labels | Transformed boxes and labels | Transformed boxes and labels |
Below the images, it would show the annotation checks and collapsible policy/trace details. No model training, external service, or DVC dependency would be required.
## Why this fits CML
- **CML** owns the CI and pull-request review surface: it publishes the generated evidence where the code change is being discussed.
- **AlbumentationsX** owns synchronized image/box/label transformation, repeatable sample realizations, execution details, and policy serialization.
- **The user** gets an early, reviewable answer to “does this augmentation change still produce valid labeled training samples?”
The existing tabular getting-started example would remain unchanged. This could be a standalone advanced guide in `cml.dev` or a separate focused example repository, whichever is easier for the maintainers to own.
## Dependency boundary
- The public `albumentationsx` package is AGPL-3.0-only and imports as `albumentations`.
- Importing it requires PyTorch. PyTorch is intentionally not selected by the package metadata because users need to choose the appropriate CPU, CUDA, or MPS build.
- These dependencies would remain scoped to the example and would not become CML runtime dependencies.
I searched the current `iterative/cml` and `iterative/cml.dev` code, Issues, and pull requests and found no existing Albumentations or AlbumentationsX integration or overlapping augmentation-review proposal.
If this use case fits the CML documentation, I would be happy to implement and run the example after the maintainers confirm the preferred placement.
Contributor guide
Research direction
Start by reviewing the existing tabular getting-started example in cml.dev and confirm whether this belongs in the docs or a standalone example repository. Then trace the proposed review_augmentation.py, build_report.py, and GitHub Actions workflow, including the AlbumentationsX policy, generated artifacts, and CML publication step. Done means a focused example produces reviewable images, checks, trace details, policy JSON, and a pull-request report without model training or new CML runtime dependencies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python, pytorch
- Domain
- ci-cd, documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100