Support BatchMM for `aiter.gemm_a8w8_blockscale()`

Open
#472 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
machine-learning

Research direction

Start by inspecting the implementations and callers of aiter.gemm_a8w8_blockscale() and aiter.batched_gemm_a8w8_CK(). Determine how BatchMM for W8A8 blockscale should differ from the existing batched interface; the issue does not specify an entry point, tests, or acceptance criteria.

Written by the indexing model from the issue text.

Description

Hi, I'd like to run with a batch of Gemm in W8A8 format, does any aiter interface capable of this purpose?

Note that this API aiter.batched_gemm_a8w8_CK() is not for blockscale.

Dominant language
Python
Stars
565
Forks
585
Avg merge
3d 4h
Merged PRs (30d)
366

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ROCm/aiter

All issues in ROCm/aiter

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.