QuantEcon / QuantEcon/lecture-python.myst

[JAX] Remove `@jax.jit` on intermediate functions

Open
#719 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TeX
Stars
123
Forks
57
Avg merge
3d 10h
Merged PRs (30d)
11

Description

From @jstac:

Adding @jax.jit to intermediate functions that are only called from within other jitted functions can actually hurt performance in several ways:

  1. Prevents fusion: JAX's compiler (XLA) works best when it can see the entire computation graph at once. When you jit intermediate
   functions separately, you create compilation boundaries that prevent the compiler from optimizing across those boundaries (like
  fusing operations, eliminating intermediate arrays, etc.)
  2. Multiple compilation overhead: Each @jax.jit decorator triggers a separate compilation, and when you call a jitted function from
   another jitted function, JAX has to manage multiple compiled kernels instead of one optimized kernel.
  3. Missed optimization opportunities: The XLA compiler can do things like:
    - Fuse element-wise operations
    - Eliminate temporary arrays
    - Optimize memory layout
    - Reorder operations for better cache usage

  But only if it can see all the operations together in one compilation unit.
  4. Dispatch overhead: Calling from one jitted function to another jitted function adds small dispatch costs that wouldn't exist if
  everything was compiled together.

  The rule of thumb: Only use @jax.jit on the "top-level" functions that users call directly. Let the inner helper functions be
  compiled as part of the larger computation graph.
  • Examine current code to see if they are following the best practice on using @jax.jit.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining the current code for uses of @jax.jit and identify intermediate functions that are called only from other jitted functions. Compare those uses with the top-level-function rule described in the issue. Done means unnecessary intermediate decorators are removed while appropriate top-level JAX compilation remains.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.