SAM3 mask exemplars: claimed in the paper, unusable in the release

**TL;DR.** The SAM3 paper/README advertise masks as a usable visual prompt for the detector ("text or visual prompts such as points, boxes, and **masks**"; detector "conditioned on text, geometry, and image exemplars"). In the released artifact, mask exemplars for the open-vocabulary **detector** are non-functional at every layer. From the code: the config never builds the mask encoder, the checkpoint has no mask-encoder weights, and the public API has no mask entry point. Box exemplars, by contrast, are fully trained and live.

---

## The claim

`README.md`:

- L49: *"It can detect, segment, and track objects using text or visual prompts such as
points, boxes, and **masks**."*
- L188: *"The detector is a DETR-based model conditioned on text, geometry, and image
**exemplars**."*

So the paper presents masks as a first-class prompt to the concept detector.

---

## The gap, layer by layer

### 1. Config never instantiates the mask encoder

`model_builder.py:_create_geometry_encoder()` (L235–288) is the **only** geometry-encoder
builder (used at both L616 and L688). It constructs `SequenceGeometryEncoder` with points
and boxes projections only:

```python
input_geometry_encoder = SequenceGeometryEncoder(
...
points_direct_project=True, points_pool=True, points_pos_enc=True,
boxes_direct_project=True, boxes_pool=True, boxes_pos_enc=True,
...
) # no mask_encoder=, no add_mask_label=
```

`mask_encoder` and `add_mask_label` therefore fall to their defaults `None` / `False`
(`geometry_encoders.py:507-508`). In any instantiated model `self.mask_encoder is None`,
so the mask branches at `geometry_encoders.py:800` and `:834` are skipped, and
`Prompt.append_masks` (`:366`) feeds nothing downstream.

### 2. The checkpoint has no mask-encoder weights

Dumping `sam3.pt` (1465 params), `detector.geometry_encoder.*` contains **only**:

- `points_*` (direct_project, pool_project, pos_enc_project)
- `boxes_*` (direct_project, pool_project, pos_enc_project)
- `cls_embed`, `label_embed`, `encode.{0,1,2}.*`, `encode_norm`, `final_proj`, `norm`,
`img_pre_norm`

There is **no** `detector.geometry_encoder.mask_encoder.*` and **no** `mask_label_embed`.
The only `mask_downsampler` anywhere in the checkpoint is
`tracker.maskmem_backbone.mask_downsampler.*` — the SAM2-lineage video-memory encoder, a
different module on the tracker, not the detector. `exemplar` and `visual_prompt` appear
**zero** times in the weights.

Consequence: even if you wired a `FusedMaskEncoder` into the config, it would be randomly
initialized. The capability cannot be recovered from the released weights without training.

### 3. The public API has no mask entry point

`Sam3Processor` (`sam3_image_processor.py`) exposes only:

- `set_text_prompt` (L113)
- `add_geometric_prompt` (L128–150) — **box-only**, via `append_boxes`

There is no method to add a mask exemplar, despite `Prompt.append_masks`
(`geometry_encoders.py:366`) and `_encode_masks` (`:683`) being fully implemented.

### 4. The detector's only mask-encoder reference is dead code

`sam3_image.py:425 _get_best_mask` calls
`self.geometry_encoder.mask_encoder.mask_downsampler(...)`, which would `AttributeError` on
`None`. The method is never called anywhere in the repo, so it never fires.

---

## Net

The mask-exemplar feature ships as inert scaffolding — `FusedMaskEncoder`, `_encode_masks`,
`Prompt.append_masks`, `Prompt.mask_embeddings`, `_get_best_mask` — that is not instantiated
by the config, not trained in the checkpoint, and not reachable from the API. Unlike box
exemplars (trained and live via `geometry_encoder`), mask exemplars cannot be used without
training a mask encoder and re-exposing it through the processor.

---

### Reproduction

Checkpoint key inspection without torch (`.pt` is a zip; param names are plain strings in
the pickle):

```python
import zipfile, re
z = zipfile.ZipFile('sam3_repo/sam3.pt')
data = z.read([n for n in z.namelist() if n.endswith('data.pkl')][0])
strs = [s.decode() for s in re.findall(rb'[ -~]{4,}', data)]
print(sorted({s for s in strs if 'mask_encoder' in s or 'mask_label' in s}))
# -> only tracker.maskmem_backbone.mask_downsampler.* (no detector.geometry_encoder.mask_encoder.*)
print(sorted({s for s in strs if 'exemplar' in s.lower() or 'visual_prompt' in s.lower()}))
# -> []
```

Start with model_builder.py:_create_geometry_encoder(), geometry_encoders.py:507-508 and the cited mask branches to trace why detector mask prompts are skipped. Run the provided sam3.pt key inspection, then review Sam3Processor in sam3_image_processor.py and sam3_image.py:_get_best_mask. Done requires a trained detector mask encoder in the checkpoint and a usable processor entry point for mask exemplars.

Written by the indexing model from the issue text.