facebookresearch / facebookresearch/sam2

Support non-RGB inference (single-channel / grayscale) inputs without forking

Open
#751 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
19.9k
Forks
2.5k
PR merge metrics
No merged PRs in 30d

Description

## Feature Request

SAM2 is increasingly applied to non-RGB modalities — medical imaging (CT/MRI), satellite imagery, infrared, depth maps — but the codebase currently hardcodes 3-channel RGB assumptions in several places, requiring users to fork and patch the repo to use single-channel inputs.

## Affected locations

- `sam2/modeling/backbones/hieradet.py` — `PatchEmbed` receives no `in_chans` param from `Hiera`
- `sam2/utils/misc.py` — `_load_img_as_tensor` hardcodes `.convert("RGB")`, and frame tensor allocation hardcodes `3`
- `sam2/sam2_video_predictor.py` — `img_mean`/`img_std` are not configurable at the predictor level

## Proposed fix

PR #749 adds backwards-compatible parameters (`in_chans`, `img_mean`, `img_std`) with unchanged defaults so existing RGB workflows are unaffected.

## Minimal usage (grayscale)

```python
predictor = SAM2VideoPredictor(
...,
img_mean=(0.5,),
img_std=(0.5,),
)

trunk:
_target_: sam2.modeling.backbones.hieradet.Hiera
in_chans: 1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.