huggingface / huggingface/pytorch_block_sparse
Using a smaller block size
- Dominant language
- C++
- Stars
- 551
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
First of all thanks for setting up this package :) It's super helpful, thanks
I'm wondering, is there a way to use a smaller block size ? I tried modifying the python code so that no errors are thrown, however I'm hitting a
`RuntimeError: CUDA error: an illegal memory access was encountered`
error when calling the cuda kernel. I tried to look a bit into the kernel code, and it seems that the `block_size` argument [is not used](https://github.com/huggingface/pytorch_block_sparse/blob/e71b54273a25ecebd2aa57181d87664e5de98dc3/pytorch_block_sparse/native/block_sparse_cutlass_kernel.cu#L121). So I'm curious how the kernel knows to expect a minimal size of 32.
Any clarifications would be super helpful!
Thanks
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.