pytorch / pytorch/pytorch

No hash in CachingAutotuner meta information for some kernels

Open
#171,184 5 comments 0 reactions 0 assignees View on GitHub
module: inductor oncall: pt2 triaged
Dominant language
Python
Stars
103k
Forks
29.5k
PR merge metrics
PR metrics pending

Description

### 🐛 Describe the bug

While working on [this issue](https://github.com/pytorch/pytorch/issues/166949) I encountered a bug that happens for templated kernels. For some reason the kernel gets compiled and runs just fine but it's missing the kernel hash and file path in the inductor meta information structures that are used by CachingAutotuner. You can check this by running such kernel with `TORCHINDUCTOR_DUMP_LAUNCH_PARAMS=1`. This will NOT produce any files in this kernel's cash folder (no bestconfig as well) apart from the kernel python code. This happens due to `self.filename, self.kernel_hash, self.fn.__name__` all being empty for CachingAutotuner for this kernel.

Attaching python repros for with Inductor meta for H100 and gfx942. Can't make a general one since inductor requires meta data for the arch it runs on, can be easily changed in the file itself if you run it on a different arch.

[repro_unhashed_kernel_h100.py](https://github.com/user-attachments/files/24311316/repro_unhashed_kernel_h100.py)
[repro_unhashed_kernel_gfx942.py](https://github.com/user-attachments/files/24311317/repro_unhashed_kernel_gfx942.py)

Steps to reproduce:

1. Grab nightly pytorch image.
2. Runs the file with `TORCHINDUCTOR_DUMP_LAUNCH_PARAMS=1`.
3. Check inductor cache folder (torchinductor_root usually) which should have one folder for this kernel with missing bestconfig.

### Versions

Reproducible on ToT Pytorch, Triton version is irrelevant.

cc @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @aakhundov @coconutruben @jataylo

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.