facebookresearch / facebookresearch/sam2
Fine-tuning SAM2 model with Eval dataset
- Dominant language
- Jupyter Notebook
- Stars
- 19.9k
- Forks
- 2.5k
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
Thanks for sharing scripts for SAM model fine-tuning. The usage example is extremely helpful.
However, there's one thing which seems a little bit off: when I try to use `val` by simply copying the`train` part in the config [here](https://github.com/facebookresearch/sam2/blob/main/sam2/configs/sam2.1_training/sam2.1_hiera_b%2B_MOSE_finetune.yaml#L203), I encounter an error at the time of assertion ([here](https://github.com/facebookresearch/sam2/blob/2b90b9f5ceec907a1c18123530e92e794ad901a4/training/trainer.py#L971)), because `set(["all"])` is being excluded from the loss keys. What's the intended way of adding an evaluation stage to the training?
My updated config looks like this:
```
trainer:
...
mode: train
...
data:
train:
...
val:
_target_: training.dataset.sam2_datasets.TorchTrainMixedDataset
batch_sizes:
- ${scratch.train_batch_size}
datasets:
- _target_: training.dataset.utils.RepeatFactorWrapper
dataset:
_target_: training.dataset.utils.ConcatDataset
datasets:
- _target_: training.dataset.vos_dataset.VOSDataset
transforms: ${vos.train_transforms}
training: false
video_dataset:
_target_: training.dataset.vos_raw_dataset.PNGRawDataset
img_folder: ${dataset.img_folder}
gt_folder: ${dataset.gt_folder}
file_list_txt: ${dataset.val_file_list_txt}
sampler:
_target_: training.dataset.vos_sampler.RandomUniformSampler
num_frames: ${scratch.num_frames}
max_num_objects: ${scratch.max_num_objects}
multiplier: ${dataset.multiplier}
num_workers: ${scratch.num_train_workers}
pin_memory: True
shuffle: False
drop_last: False
collate_fn:
_target_: training.utils.data_utils.collate_fn
_partial_: true
**dict_key: all**
```
Should the loss and collate_fn be different for the eval stage?
Contributor guide
Assessment
This issue has not been assessed yet.