Proposal: publish Gemma conformance test vectors

## Proposal

Publish a small, versioned set of **Gemma conformance test vectors** that downstream implementations can run to verify behavioral compatibility with the DeepMind reference implementation.

The fixtures would be compact reference cases covering semantics that are easy to implement almost correctly but still diverge in hard-to-diagnose ways.

A first version could cover:

- tokenizer input/output IDs;
- attention masks and positions;
- prefill logits for fixed inputs;
- incremental decode steps, including KV-cache positions and expected logits;
- multimodal preprocessing where applicable;
- DiffusionGemma denoising/self-conditioning state for a few fixed steps and masks.

Each fixture could contain the model/config identifier, small inputs, selected intermediate metadata, and reference outputs with documented numerical tolerances. The harness would load the normal published checkpoint rather than storing model weights in the repository.

## Motivation

Gemma is implemented and served through multiple ecosystems. When implementations disagree, it is often unclear whether the difference comes from tokenization, masking, positions, cache indexing, diffusion state handling, or ordinary numerical tolerance.

A first-party conformance suite would give implementers a precise compatibility target and make regressions easier to localize.

## Possible structure

```text
conformance/
gemma3/
tokenizer.json
prefill.json
decode_step.json
diffusion_gemma/
denoise_step_0.json
denoise_step_1.json
selected_canvas.json
```

with a small runner such as:

```bash
python -m gemma.conformance --model gemma3 --checkpoint /path/to/checkpoint
```

The runner would report each semantic boundary separately rather than only comparing a final generated string.

## Scope

I would keep the first contribution deliberately narrow:

1. define a versioned fixture schema;
2. add a runner for the existing reference implementation;
3. publish a handful of deterministic Gemma/DiffusionGemma vectors;
4. document tolerance rules for external implementations;
5. add CI coverage ensuring the reference implementation continues to reproduce them.

No external framework dependency would be required.

I searched current issues and PRs for conformance vectors, golden fixtures, implementation parity, and reference logits and did not find an equivalent proposal.

If this direction is useful, I would be happy to prototype the fixture format, runner, tests, and an initial set of vectors.

Start by inspecting the existing reference implementation and the proposed `conformance/` fixture layout, then review the `python -m gemma.conformance` entry point described in the issue. Define the versioned schema, runner, deterministic Gemma and DiffusionGemma vectors, tolerance rules, and CI coverage. Done means downstream implementations have documented fixtures and the reference implementation reproduces them.

Written by the indexing model from the issue text.