facebookresearch / facebookresearch/sam2
Data Transform for fine-tuning (image & video datasets)
- Dominant language
- Jupyter Notebook
- Stars
- 19.9k
- Forks
- 2.5k
- PR merge metrics
- No merged PRs in 30d
Description
Hi, have anyone ever encountered issues with "transforms"? Whether fine-tuning on a video or image dataset, I don't quite understand the importance of each transform. It seems that the only essential ones (if removed, errors occur) are RandomResize, ToTensor, and Normalize. Can the others be omitted?
`vos:
train_transforms:
- _target_: training.dataset.transforms.ComposeAPI
transforms:
# - _target_: training.dataset.transforms.RandomHorizontalFlip
# consistent_transform: True
# - _target_: training.dataset.transforms.RandomAffine
# degrees: 25
# shear: 20
# image_interpolation: bilinear
# consistent_transform: True
- _target_: training.dataset.transforms.RandomResizeAPI
sizes: ${scratch.resolution}
square: true
consistent_transform: True
# - _target_: training.dataset.transforms.ColorJitter
# consistent_transform: True
# brightness: 0.1
# contrast: 0.03
# saturation: 0.03
# hue: null
# - _target_: training.dataset.transforms.RandomGrayscale
# p: 0.05
# consistent_transform: True
# - _target_: training.dataset.transforms.ColorJitter
# consistent_transform: False
# brightness: 0.1
# contrast: 0.05
# saturation: 0.05
# hue: null
- _target_: training.dataset.transforms.ToTensorAPI
- _target_: training.dataset.transforms.NormalizeAPI
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]`
Contributor guide
Research direction
The issue names the training.dataset.transforms entry points and a vos.train_transforms configuration, but no source file or test. Start by locating the listed transform implementations and their dataset/configuration usage; document what each transform does, which are required, and how to verify image and video fine-tuning still works.
Written by the indexing model from the issue text.
Assessment
- Domain
- data, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100