facebookresearch / facebookresearch/sam2
Support non-RGB inference (single-channel / grayscale) inputs without forking
- Dominant language
- Jupyter Notebook
- Stars
- 19.9k
- Forks
- 2.5k
- PR merge metrics
- No merged PRs in 30d
Description
## Feature Request
SAM2 is increasingly applied to non-RGB modalities — medical imaging (CT/MRI), satellite imagery, infrared, depth maps — but the codebase currently hardcodes 3-channel RGB assumptions in several places, requiring users to fork and patch the repo to use single-channel inputs.
## Affected locations
- `sam2/modeling/backbones/hieradet.py` — `PatchEmbed` receives no `in_chans` param from `Hiera`
- `sam2/utils/misc.py` — `_load_img_as_tensor` hardcodes `.convert("RGB")`, and frame tensor allocation hardcodes `3`
- `sam2/sam2_video_predictor.py` — `img_mean`/`img_std` are not configurable at the predictor level
## Proposed fix
PR #749 adds backwards-compatible parameters (`in_chans`, `img_mean`, `img_std`) with unchanged defaults so existing RGB workflows are unaffected.
## Minimal usage (grayscale)
```python
predictor = SAM2VideoPredictor(
...,
img_mean=(0.5,),
img_std=(0.5,),
)
trunk:
_target_: sam2.modeling.backbones.hieradet.Hiera
in_chans: 1
Contributor guide
Assessment
This issue has not been assessed yet.