lablup / lablup/mlxcel

test(models): validate Inkling omni on real checkpoints

Open
#1,549 0 comments 0 reactions 0 assignees View on GitHub
arch:moe area:inference area:models modelsize:xlarge modeltype:omni platform:macos priority:medium status:backlog type:test
Dominant language
Rust
Stars
467
Forks
54
Avg merge
4h 25m
Merged PRs (30d)
310

Description

## Summary

Validate the merged Inkling omni implementation from epic #1313 on the published real checkpoints and record parity, quality, and throughput evidence that could not be produced on the Linux validation host.

## Context

The implementation is merged through PRs #1532, #1535, #1540, #1546, and #1548. Their deterministic CPU/reference tests, synthetic MLX tests, CI, lint, and contract checks pass. The available host did not have the approximately 153.5 GB affine Inkling-Small checkpoint, the approximately 170.7 GB native NVFP4 checkpoint, the 0.6B community checkpoint, or a compatible Apple GPU with sufficient unified memory.

## Validation matrix

- [ ] On Apple Silicon with at least 192 GB unified memory, load `mlx-community/Inkling-Small-mlx-4bit` and produce 64 greedy text tokens through the CLI and `/v1/chat/completions` using the checkpoint chat template.
- [ ] Compare the 0.6B checkpoint against the upstream reference for a RAW prompt, greedy decoding, 64 tokens, and `MLXCEL_FUSED_MOE=0`; require token-exact text parity.
- [ ] Run a grounded still-image request through both the CLI and server.
- [ ] Transcribe a clean five-second 16 kHz WAV through both the CLI and server; require placeholder count `ceil(samples / 800)` and word error rate below 20 percent.
- [ ] Describe motion in a synthetic left-to-right video through the adjacent-frame path; record sampled anchors, timestamps, and token count.
- [ ] Load the native MTP shard, verify 128-token greedy equality with plain decoding, record mean accepted length, and require a positive decode-throughput improvement.
- [ ] Validate the native NVFP4 checkpoint, including expert sidecar scales, on a reachable compatible host.

## Evidence to attach

- Exact model revisions and file hashes
- Hardware and OS details
- Commands and complete outputs
- Token sequences or parity diffs
- Audio transcript and word error rate
- Video sampling metadata and answer
- Plain versus MTP throughput and acceptance statistics

## Related

- Epic #1313
- Text #1318 / PR #1532
- Vision #1327 / PR #1535
- MTP #1315 / PR #1540
- Video #1323 / PR #1546
- Audio #1311 / PR #1548

Contributor guide

Open the contributing guide

Research direction

Start by reading the merged validation work in PRs #1532, #1535, #1540, #1546, and #1548, then exercise the CLI and /v1/chat/completions entry points on the specified Apple Silicon host. Done means completing the validation matrix with exact model revisions, hashes, hardware details, full outputs, parity data, quality metrics, video metadata, and plain-versus-MTP throughput evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, cli, machine-learning, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.