FusedMHARunnerV2 function mSM check
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 14.7k
- Forks
- 2.8k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 489
Description
System Info
FusedMHARunnerV2::FusedMHARunnerV2(MHARunnerFixedParams fixedParams)
: mFixedParams(fixedParams)
{
TLLM_CHECK_WITH_INFO((mSM == kSM_80 || mSM == kSM_86 || mSM == kSM_89 || mSM == kSM_90 || mSM == kSM_120),
"Unsupported architecture");
TLLM_CHECK_WITH_INFO((mFixedParams.dataType == DATA_TYPE_FP16 || mFixedParams.dataType == DATA_TYPE_BF16
|| mFixedParams.dataType == DATA_TYPE_E4M3),
"Unsupported data type");
xmmaKernel = getXMMAKernelsV2(mFixedParams.dataType, mSM);
if (mFixedParams.headSizeV == 0)
{
mFixedParams.headSizeV = mFixedParams.headSize;
}
// Get device attributes.
int device_id;
cudaGetDevice(&device_id);
cudaDeviceGetAttribute(&mMultiProcessorCount, cudaDevAttrMultiProcessorCount, device_id);
cudaDeviceGetAttribute(&mDeviceL2CacheSize, cudaDevAttrL2CacheSize, device_id);
auto const [free_memory, total_memory] = tensorrt_llm::common::getDeviceMemoryInfo(false);
mTotalDeviceMemory = total_memory;
}
can tensorr-llm only support kSM_80 kSM_86 kSM_89 kSM_90 kSM_120?Can tensorr-llm support kSM_87?The support plan?
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
NO
Expected behavior
NO
actual behavior
nothing
additional notes
nothing
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with FusedMHARunnerV2::FusedMHARunnerV2 and its mSM validation, then trace getXMMAKernelsV2 for the listed architectures. Determine whether kSM_87 has supported kernels and what the project’s support policy requires; done means a clear support decision and an agreed implementation or documentation path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- ai-infra-agents
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100