facebookresearch / facebookresearch/sam2

Data Transform for fine-tuning (image & video datasets)

Open
#621 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

Hi, have anyone ever encountered issues with "transforms"? Whether fine-tuning on a video or image dataset, I don't quite understand the importance of each transform. It seems that the only essential ones (if removed, errors occur) are RandomResize, ToTensor, and Normalize. Can the others be omitted?
`vos:
train_transforms:
- _target_: training.dataset.transforms.ComposeAPI
transforms:
# - _target_: training.dataset.transforms.RandomHorizontalFlip
# consistent_transform: True
# - _target_: training.dataset.transforms.RandomAffine
# degrees: 25
# shear: 20
# image_interpolation: bilinear
# consistent_transform: True
- _target_: training.dataset.transforms.RandomResizeAPI
sizes: ${scratch.resolution}
square: true
consistent_transform: True
# - _target_: training.dataset.transforms.ColorJitter
# consistent_transform: True
# brightness: 0.1
# contrast: 0.03
# saturation: 0.03
# hue: null
# - _target_: training.dataset.transforms.RandomGrayscale
# p: 0.05
# consistent_transform: True
# - _target_: training.dataset.transforms.ColorJitter
# consistent_transform: False
# brightness: 0.1
# contrast: 0.05
# saturation: 0.05
# hue: null
- _target_: training.dataset.transforms.ToTensorAPI
- _target_: training.dataset.transforms.NormalizeAPI
mean: [0.485, 0.456, 0.406]
std: [0.229, 0.224, 0.225]`

Contributor guide

Open the contributing guide

Research direction

The issue names the training.dataset.transforms entry points and a vos.train_transforms configuration, but no source file or test. Start by locating the listed transform implementations and their dataset/configuration usage; document what each transform does, which are required, and how to verify image and video fine-tuning still works.

Written by the indexing model from the issue text.

Assessment

Domain
data, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.