facebookresearch / facebookresearch/sam2

Fine-tuning SAM2 model with Eval dataset

Open
#624 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

Hello,

Thanks for sharing scripts for SAM model fine-tuning. The usage example is extremely helpful.

However, there's one thing which seems a little bit off: when I try to use `val` by simply copying the`train` part in the config [here](https://github.com/facebookresearch/sam2/blob/main/sam2/configs/sam2.1_training/sam2.1_hiera_b%2B_MOSE_finetune.yaml#L203), I encounter an error at the time of assertion ([here](https://github.com/facebookresearch/sam2/blob/2b90b9f5ceec907a1c18123530e92e794ad901a4/training/trainer.py#L971)), because `set(["all"])` is being excluded from the loss keys. What's the intended way of adding an evaluation stage to the training?

My updated config looks like this:

```

trainer:
...
mode: train
...

data:
train:
...
val:
_target_: training.dataset.sam2_datasets.TorchTrainMixedDataset
batch_sizes:
- ${scratch.train_batch_size}

datasets:
- _target_: training.dataset.utils.RepeatFactorWrapper
dataset:
_target_: training.dataset.utils.ConcatDataset
datasets:
- _target_: training.dataset.vos_dataset.VOSDataset
transforms: ${vos.train_transforms}
training: false
video_dataset:
_target_: training.dataset.vos_raw_dataset.PNGRawDataset
img_folder: ${dataset.img_folder}
gt_folder: ${dataset.gt_folder}
file_list_txt: ${dataset.val_file_list_txt}
sampler:
_target_: training.dataset.vos_sampler.RandomUniformSampler
num_frames: ${scratch.num_frames}
max_num_objects: ${scratch.max_num_objects}
multiplier: ${dataset.multiplier}
num_workers: ${scratch.num_train_workers}
pin_memory: True
shuffle: False
drop_last: False
collate_fn:
_target_: training.utils.data_utils.collate_fn
_partial_: true
**dict_key: all**

```

Should the loss and collate_fn be different for the eval stage?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.