OpenPipe / OpenPipe/ART

enforce_eager=True not being respected in test scripts

Open
#262 2 comments 0 reactions 1 assignee View on GitHub

@bradhilton is already working on this.

Since Jul 21, 2025.

bug
Dominant language
Python
Stars
10.8k
Forks
989
Avg merge
6h 29m
Merged PRs (30d)
85

Description

Problem

When running test scripts with enforce_eager=True specified, the logs still show enforce_eager=False and CUDA graphs are being calculated. This makes startup slower and leads to a slower feedback cycle during testing.

Reproduction

In the test script src/art/test/test_step_skipping.py, we're passing enforce_eager=True:

# Register the model
await model.register(
    backend,
    _openai_client_config={"engine_args": {"enforce_eager": True}},
)

However, when running the script, the logs show that enforce_eager is still False and CUDA graphs are being compiled.

Expected Behavior

When enforce_eager=True is passed in the configuration, it should:

  1. Skip CUDA graph compilation
  2. Start up faster
  3. Provide quicker feedback during testing

Impact

This issue affects development velocity as tests take longer to start and provide feedback than necessary.

Environment

  • The issue can be reproduced by running: ./src/art/test/test_step_skipping.py
  • The script is configured to run on GPU with sky launch

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.