NVIDIA / NVIDIA/cutlass

[Feature] cute.compile hardcodes no_cache=True, disabling the DSL's content-addressed compile cache

Open Beginner friendly
#3,398 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

Summary

The explicit compile entry point in the CuTeDSL Python runtime unconditionally forces
no_cache=True, so the DSL's own content-addressed on-disk MLIR cache never engages for
cute.compile. Every process that compiles the same kernel re-runs the full MLIR build from
scratch, even though a byte-identical artifact already exists on disk.

Where

python/CuTeDSL/cutlass/base_dsl/compiler.py, in _compile() (the shared path behind
cute.compile):

# compiler.py:1318-1319
kwargs["compile_only"] = True
kwargs["no_cache"] = True

no_cache is set on every call, overriding anything the caller passed and bypassing the
cache machinery the DSL otherwise maintains. There is no argument or environment override that
reaches this line.

Observed on nvidia-cutlass-dsl 4.5.2; the same two lines are present on main (4.6).

Impact

We serve an inference engine whose attention kernels are @cute.jit functions compiled via
explicit cute.compile at startup. Because no_cache=True is forced, each server process pays
the full MLIR compile on every boot:

  • Cold boot (no cache possible): ~506 s to ready.
  • With the DSL cache re-enabled externally (we wrap generate_mlir to clear the forced flag):
    warm boot ~58 s — an ~8.7× boot-time reduction, with the reloaded artifacts verified
    byte-identical (same on-disk files, unchanged mtimes, zero new writes).

So the cache the DSL already implements recovers almost all of the compile cost — it's just
switched off at this one call site. Working around it requires wrapping an internal method whose
signature is not a stable API, which is fragile across DSL releases.

Requested change

Stop unconditionally forcing no_cache=True in _compile(). Any of these would resolve it:

  1. Honor a caller-supplied no_cache (only default it when unset), or
  2. Gate it on an environment variable (e.g. CUTE_DSL_COMPILE_CACHE=1), consistent with the
    existing CUTE_DSL_* option surface, or
  3. Enable the content-addressed cache by default for the explicit cute.compile path — its key
    already includes traced IR + target arch + toolkit version, so stale replay is not possible.

Illustrative sketch:

-        kwargs["compile_only"] = True
-        kwargs["no_cache"] = True
+        kwargs["compile_only"] = True
+        kwargs.setdefault("no_cache", _env_default_no_cache())  # respect caller / CUTE_DSL_* env
Reproduce

Compile any @cute.jit function via cute.compile twice, in two fresh processes, with a
persistent CUTE_DSL cache directory set. The second process still performs a full MLIR compile
rather than a cache hit, because _compile() forces no_cache=True. Happy to provide a
self-contained repro script.


cc @Junkai-Wu — you appear as the author on the CuTeDSL mirror drops (e.g. #3362); apologies if
this isn't your area — if so, could you help route it to whoever owns the DSL compiler path? Thanks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in python/CuTeDSL/cutlass/base_dsl/compiler.py at _compile(), then read the existing cache machinery and how no_cache is passed through cute.compile. Reproduce the two-process compile with a persistent CUTE_DSL cache directory; done means a repeated compilation reuses the content-addressed artifact without rebuilding or writing new cache files.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers, performance
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.