openai / openai/codex

Clarify test expectations for latest-alpha-cli releases

Open
#40,240 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What is the expected test policy for alpha releases published from latest-alpha-cli, and what should contributors do if post-merge CI finds a regression?

The workflow documentation says that Cargo-native PR checks are intentionally small and that the full Cargo nextest matrix runs post-merge. The release workflow updates latest-alpha-cli after the release job, but I do not see the full test workflow among that job's dependencies. Is there an additional release gate or policy that is not documented here?

A full-suite-passing alpha would provide downstream contributors with a known-good baseline: when a test fails after their change, they could more quickly identify their change as the cause. Without that baseline, they first need to run the full suite against an unmodified alpha to determine whether the failure already exists upstream.

For a concrete example, PR #39372 left an existing TUI test failing because its test recorder changed; the regression is described in #40238. The production change appears valid, but the alpha branch can contain a failing test unless the full suite is run and acted on before publication.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read .github/workflows/README.md and .github/workflows/rust-release.yml, then compare the documented Cargo nextest workflow with the release job dependencies. Confirm the expected alpha-release gate and post-merge regression response with maintainers; done means the policy and any required workflow relationship are documented clearly.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, rust
Domain
ci-cd, release
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.