pytorch / pytorch/pytorch

Inductor custom pass mechanism that maintains config constraints

Open
#165,501 2 comments 0 reactions 0 assignees View on GitHub
module: inductor module: vllm oncall: pt2 triaged vllm-compile
Dominant language
Python
Stars
103k
Forks
29.5k
PR merge metrics
PR metrics pending

Description

From discussion with @oulgen . We require configs to be serializable. The inductor custom pass mechanism is to put a stateful object into the inductor config. The stateful object is not always serializable. We should have a better custom pass mechanism that isn't putting a stateful object into the inductor config, then migrate vLLM to using it so we stop triggering this problem.

cc @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @aakhundov @coconutruben

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.