lightly-ai / lightly-ai/lightly
Tutorial proposal: customize and inspect DINO views with AlbumentationsX
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 367
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 5
Description
Would you be open to a small standalone tutorial showing how to change DINO global and local view policies and inspect a selected realization without copying Lightly's full `DINOTransform`?
## Motivation
Issue #1814 documents a concrete limitation: changing the `RandomResizedCrop` ratio currently requires reimplementing most of the DINO transform. The maintainer discussion there also considers a more composable transform abstraction, while noting that the refactor is not currently prioritized.
A focused tutorial could give users a documented composition pattern with the current Lightly API. Lightly would continue to define the DINO view roles, dataset contract, model, and training workflow. AlbumentationsX would define the image pipeline for each global or local view.
The user could then change crop ratio, scale, or photometric operations directly, save each configured policy with `A.to_dict()`, and reproduce one diagnostic view with `run_with_trace(..., invocation_seed=...)` to inspect which transforms ran and which parameters they sampled.
## Proposed tutorial
The tutorial would:
1. Define separate AlbumentationsX pipelines for DINO's two global views and its local views, including the intended blur and solarization differences.
2. Adapt each pipeline from a PIL image to the tensor returned by `A.ToTensorV2()`.
3. Pass those callables to Lightly's `MultiViewTransform` and use the normal Lightly training contract.
4. Serialize the configured view policies with `A.to_dict()`.
5. Run one selected image through `run_with_trace()` with explicit per-view `invocation_seed` values, so the same diagnostic views and sampled parameters can be regenerated.
A small sketch of the composition boundary is:
```python
import numpy as np
import albumentations as A
from lightly.transforms import MultiViewTransform
class AXView:
def __init__(self, pipeline: A.Compose) -> None:
self.pipeline = pipeline
def __call__(self, image):
return self.pipeline(image=np.asarray(image))["image"]
def make_view(size: int, scale: tuple[float, float]) -> A.Compose:
return A.Compose(
[
A.RandomResizedCrop(
size=(size, size),
scale=scale,
ratio=(0.7, 1.4),
p=1.0,
),
A.HorizontalFlip(p=0.5),
A.Normalize(),
A.ToTensorV2(),
]
)
global_view_0 = make_view(size=224, scale=(0.4, 1.0))
global_view_1 = make_view(size=224, scale=(0.4, 1.0))
local_view = make_view(size=96, scale=(0.05, 0.4))
transform = MultiViewTransform(
transforms=[AXView(global_view_0), AXView(global_view_1)]
+ [AXView(local_view)] * 6
)
```
The full tutorial would use the DINO-specific photometric policies; the sketch keeps only the code needed to show the Lightly–AX boundary.
## Scope and placement
This would be a documentation tutorial. It would not change `DINOTransform`, `MultiViewTransform`, or Lightly's runtime dependencies. The existing legacy Albumentations material in `structure_your_input.rst` would remain unchanged.
PR #2037 is currently exploring new view contracts for LightlySSL 2.0. Which contract should the tutorial target: the current API or the proposed 2.0 view API?
If the tutorial fits the project, I can prepare a documentation-only PR after your guidance on placement and target API.
## Optional dependency boundary
The tutorial would be explicitly optional and require Python 3.10 or newer. The public AlbumentationsX package is AGPL-3.0-only. Importing it also requires PyTorch; PyTorch is intentionally absent from its package metadata because the user must select the CPU, CUDA, or MPS build required by the application. The tutorial would document that installation boundary. AlbumentationsX would remain outside Lightly's runtime dependencies.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading structure_your_input.rst and reviewing the current MultiViewTransform API alongside PR #2037. Confirm with maintainers whether the tutorial should target the current API or the proposed 2.0 view contract, then document the separate DINO view policies, serialization, trace-based inspection, and optional dependency boundary. Done means a standalone documentation tutorial is placed appropriately without changing Lightly runtime dependencies.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100