@Abigbigbig This looks like a different issue from this PR. Let's move to a different issue. I can point you the fix

@Abigbigbig This looks like a different issue from this PR. Let's move to a different issue. I can point you the fix

Originally posted by @yubofredwang in https://github.com/sgl-project/SpecForge/issues/314#issuecomment-3588281081
Thank you for your answer，I have now reverted back to specforge 0.1.0 and sglang 0.5.4, and used your previous solution to add "kernel_options={
"BLOCK_M": 32,
"BLOCK_N": 32,
"BLOCK_M1": 32,
"BLOCK_N1": 32,
"BLOCK_M2": 32,
"BLOCK_N2": 32,
}
attn_output = flex_attention_func(
query=query_states,
key=key_cache.contiguous(),
value=value_cache.contiguous(),
block_mask=block_mask,
enable_gqa=True,
kernel_options=kernel_options,
）”Solved the compilation issue of 'OutOfCacheError: out of resource: triton-tem_fused_0 Required: 107008 Hardware limit: 101376', but there was insufficient video memory during training. I am using two 48GB A6000 graphics cards. Can this model of graphics card meet the requirements. The specific question is "[rank1]: torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.02 GiB. GPU 1 has a total capacity of 47.40 GiB of which 30.69 GiB is free. Process 583936 has 29.95 GiB memory in use. Process 1818617 has 16.71 GiB memory in use. Of the allocated memory 16.25 GiB is allocated by PyTorch, and 204.79 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)".

No repository file, test, or entry point is named; start by reviewing the referenced discussion in issue 314 and the reported PyTorch CUDA out-of-memory output. The issue itself says this is different from the original PR and should move to another issue, so a clear completion condition is not established here.