[Feature] cute.compile hardcodes no_cache=True, disabling the DSL's content-addressed compile cache
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 10.5k
- Forks
- 2.1k
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 7
Description
Summary
The explicit compile entry point in the CuTeDSL Python runtime unconditionally forces
no_cache=True, so the DSL's own content-addressed on-disk MLIR cache never engages for
cute.compile. Every process that compiles the same kernel re-runs the full MLIR build from
scratch, even though a byte-identical artifact already exists on disk.
Where
python/CuTeDSL/cutlass/base_dsl/compiler.py, in _compile() (the shared path behind
cute.compile):
# compiler.py:1318-1319
kwargs["compile_only"] = True
kwargs["no_cache"] = True
no_cache is set on every call, overriding anything the caller passed and bypassing the
cache machinery the DSL otherwise maintains. There is no argument or environment override that
reaches this line.
Observed on nvidia-cutlass-dsl 4.5.2; the same two lines are present on main (4.6).
Impact
We serve an inference engine whose attention kernels are @cute.jit functions compiled via
explicit cute.compile at startup. Because no_cache=True is forced, each server process pays
the full MLIR compile on every boot:
- Cold boot (no cache possible): ~506 s to ready.
- With the DSL cache re-enabled externally (we wrap
generate_mlirto clear the forced flag):
warm boot ~58 s — an ~8.7× boot-time reduction, with the reloaded artifacts verified
byte-identical (same on-disk files, unchanged mtimes, zero new writes).
So the cache the DSL already implements recovers almost all of the compile cost — it's just
switched off at this one call site. Working around it requires wrapping an internal method whose
signature is not a stable API, which is fragile across DSL releases.
Requested change
Stop unconditionally forcing no_cache=True in _compile(). Any of these would resolve it:
- Honor a caller-supplied
no_cache(only default it when unset), or - Gate it on an environment variable (e.g.
CUTE_DSL_COMPILE_CACHE=1), consistent with the
existingCUTE_DSL_*option surface, or - Enable the content-addressed cache by default for the explicit
cute.compilepath — its key
already includes traced IR + target arch + toolkit version, so stale replay is not possible.
Illustrative sketch:
- kwargs["compile_only"] = True
- kwargs["no_cache"] = True
+ kwargs["compile_only"] = True
+ kwargs.setdefault("no_cache", _env_default_no_cache()) # respect caller / CUTE_DSL_* env
Reproduce
Compile any @cute.jit function via cute.compile twice, in two fresh processes, with a
persistent CUTE_DSL cache directory set. The second process still performs a full MLIR compile
rather than a cache hit, because _compile() forces no_cache=True. Happy to provide a
self-contained repro script.
cc @Junkai-Wu — you appear as the author on the CuTeDSL mirror drops (e.g. #3362); apologies if
this isn't your area — if so, could you help route it to whoever owns the DSL compiler path? Thanks.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in python/CuTeDSL/cutlass/base_dsl/compiler.py at _compile(), then read the existing cache machinery and how no_cache is passed through cute.compile. Reproduce the two-process compile with a persistent CUTE_DSL cache directory; done means a repeated compilation reuses the content-addressed artifact without rebuilding or writing new cache files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- compilers, performance
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100