patrick-kidger / patrick-kidger/jaxtyping

Creating instances of `jaxtyped` dataclasses is slow

Open
#232 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
96
PR merge metrics
No merged PRs in 30d

Description

Annotating a dataclass with @jaxtyped makes creating instances of that class ~1000x slower.
This is especially problematic in cases where the entire package is jaxtyped with install_import_hook(), because it is not possible to exclude a frequently used dataclass from being jaxtyped.

Here is a small benchmark:

from dataclasses import dataclass
import time
from jaxtyping import jaxtyped
from beartype import beartype


N = 1000


class VanillaClass:

    def __init__(self, foo: str):
        self.foo = foo

@dataclass
class VanillaDataclass:

    foo: str

@jaxtyped(typechecker=beartype)
class JaxtypedClass:

    def __init__(self, foo: str):
        self.foo = foo

@jaxtyped(typechecker=beartype)
@dataclass
class JaxtypedDataclass:

    foo: str

@beartype
class BeartypeClass:

    def __init__(self, foo: str):
        self.foo = foo

@beartype
@dataclass
class BeartypeClassDataclass:

    foo: str


for c in [
    VanillaClass,
    VanillaDataclass,
    JaxtypedClass,
    JaxtypedDataclass,
    BeartypeClass,
    BeartypeClassDataclass
]:
    now = time.time_ns()
    for _ in range(N):
        c("foo")
    run_time = (time.time_ns() - now) / N
    print(f"{c.__name__:>25}: {run_time} ns")

Output:

             VanillaClass: 98.0 ns
         VanillaDataclass: 128.0 ns
            JaxtypedClass: 125.0 ns
        JaxtypedDataclass: 282535.0 ns
            BeartypeClass: 223.0 ns
        BeartypeDataclass: 243.0 ns

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the inline benchmark and comparing JaxtypedDataclass with the other class variants. Trace the interaction between @jaxtyped and @dataclass to locate the instantiation overhead. Done means jaxtyped dataclass construction is no longer roughly 1000x slower while runtime checking remains intact; add a regression benchmark or test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.