lablup / lablup/mlxcel

chore(release): refresh Linux aarch64 CUDA artifact for specialized serving

Open
#1,653 0 comments 0 reactions 0 assignees View on GitHub
platform:linux priority:high status:backlog type:chore
Dominant language
Rust
Stars
467
Forks
54
Avg merge
4h 25m
Merged PRs (30d)
310

Description

## Problem

The official v0.6.0 Linux aarch64 CUDA 13 archive is older than the specialized-serving behavior now required by the public recipes catalog:

- Text-generation recipes for Qwen, Gemma, and Llama reach the chat-completions boundary on an NVIDIA GB10.
- The archive does not expose the current dedicated embedding-serving interface needed by the Qwen3 Embedding recipe.
- A Whisper checkpoint loads and serves the audio transcription route, but `/health` continues to inspect the unavailable chat worker and remains at HTTP 503.

A current-main CUDA build compiled for SM 12.1 passes all five seed recipes through their task-specific HTTP endpoints on the same GB10. That proves the source behavior, but it must not be published as verification of the older v0.6.0 archive.

## Proposed work

- Publish a new versioned Linux aarch64 CUDA 13 release archive after the specialized embedding and audio-health changes reach the public repository.
- Exercise the exact downloaded archive rather than an in-tree build.
- Feed the resulting rendering hashes and logs into the downstream recipes verified-badge workflow.

Do not replace the existing v0.6.0 asset in place; a new version keeps download provenance and cached artifacts unambiguous.

## Acceptance criteria

- [ ] The new archive contains aarch64 ELF binaries dynamically linked to the expected CUDA 13 libraries with no missing shared objects on GB10.
- [ ] `mlxcel-server` from that archive starts a Qwen3 embedding checkpoint in dedicated embedding mode and `/v1/embeddings` returns a non-empty vector.
- [ ] `mlxcel-server` from that archive starts Whisper, `/health` reaches HTTP 200 after load, and `/v1/audio/transcriptions` returns non-empty text.
- [ ] Qwen3 4B, Gemma 4 E4B, and Llama 3.1 8B chat recipes return a non-empty `content` or `reasoning_content` response.
- [ ] The exact release version and binary checksum are preserved with the five-seed GB10 verification evidence.

Contributor guide

Open the contributing guide

Research direction

Use the exact downloaded versioned archive rather than an in-tree build. Start with the mlxcel-server entry point and exercise the five seed recipes through their listed HTTP endpoints on a GB10, checking binaries, shared libraries, responses, health, logs, rendering hashes, and the checksum. Done means a new archive passes every acceptance criterion with its version and verification evidence preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust
Domain
infrastructure, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.