lablup / lablup/mlxcel

docs(bench): speculative_bench module docs contradict its own code in three places

Open Beginner friendly
#1,691 0 comments 0 reactions 0 assignees View on GitHub
priority:low status:ready type:docs
Dominant language
Rust
Stars
467
Forks
54
Avg merge
4h 25m
Merged PRs (30d)
310

Description

## Summary

Correct three passages in `speculative_bench`'s module docs and `long_about` that contradict the code in the same file: a stale target count, a batch-sweep claim the code disowns, and a "deferred" claim about a path that already records real numbers.

## Background

The doc comments were written against an earlier version of the file and drifted as the pairing table and the MTP path grew. A reader trusting the header comes away with the wrong set of reachable targets, the wrong batch behavior, and the wrong idea that MTP is not yet measured.

## Proposed Solution

Edit the three passages so the module docs, `long_about`, and the pairing table describe what the code does. No run required.

## Implementation Notes

- `src/bin/speculative_bench.rs:21` claims the two reachable targets `models/qwen3.5-4b-4bit` and `models/gemma-4-31b-it-4bit`; `REACHABLE_PAIRINGS` actually lists four targets, adding `gemma-4-12b-it-4bit` (`:279`) and `qwen3.8-27b-4bit` (`:297`).
- `:61` says "Full sweep across pairings (B = 1, 2, 4)", but the sweep logs benching at B=1 and `--batch`'s own doc notes the baseline path always runs B=1 today.
- `:118-121` `long_about` says speculative paths are "scaffolded but deferred to follow-up", which contradicts the module body: MTP records real decode tok/s; only DFlash is deferred.
- Distinct from #1667, the `--prompt` panic in the same binary.

## Acceptance Criteria

- [ ] Module docs, `long_about`, and the pairing table tell one consistent story matching the code.

---

## Original Suggestion

### Title: docs(bench): speculative_bench module docs contradict its own code in three places

`speculative_bench`'s module docs and `long_about` contradict the code in the same file in three places — stale target counts, a batch-sweep claim the code disowns, and a "deferred" claim about a path that ships.

## Evidence

- `src/bin/speculative_bench.rs:21` — claims "the two reachable target models (`models/qwen3.5-4b-4bit`, `models/gemma-4-31b-it-4bit`)"; `REACHABLE_PAIRINGS` (`:243-308`) lists four targets, adding `gemma-4-12b-it-4bit` (`:281`) and `qwen3.8-27b-4bit` (`:298`)
- `:61` — "Full sweep across pairings (B = 1, 2, 4)"; the sweep logs "benching {} pairings at B=1" (`:948-950`) and `--batch`'s own doc says "The baseline path always runs B=1 today" (`:139-141`)
- `:118-121` — `long_about` says "Speculative paths are scaffolded but deferred to follow-up", contradicting `:26-33` and `:94-99` (MTP records real decode tok/s; only DFlash at `:100-103` is deferred)

## Suggested fix

Update the three passages against the code in the same file. No run required.

(Distinct from #1667, the `--prompt` panic in this binary.)

## Acceptance criteria

- [ ] Module docs, `long_about`, and the pairing table tell one consistent story

Contributor guide

Open the contributing guide

Research direction

Start in src/bin/speculative_bench.rs by comparing the module docs and long_about with REACHABLE_PAIRINGS and the MTP, DFlash, and batch-sweep descriptions. Update the three contradictory passages so they match the code and pairing table; no run is required.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
Under an hour
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
92/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.