per-target --test_timeout overrides via regex_filter@value syntax
- Dominant language
- Java
- Stars
- 25.8k
- Forks
- 4.6k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 72
Description
### Description of the feature request:
`--test_timeout` currently accepts only a single integer or a comma-separated list of four integers (the short, moderate, long, eternal categories). The override is therefore global or per-category — it cannot be scoped to an individual target or a subset of targets from the command line.
By contrast, both `--runs_per_test` and `--flaky_test_attempts` already support a `regex_filter@value` syntax that scopes the flag to targets matching a label regex, and either flag may be specified multiple times. For example:
```
--runs_per_test=^//pizza:.*@4
--flaky_test_attempts=//foo/.*,-//foo/bar/.*@3
```
I propose that `--test_timeout` could accept the same `regex_filter@value` form, so a timeout can be raised (or lowered) for specific targets without editing `BUILD` files or having to rebase. For example:
```
# Give everything under //integration/ a 600s timeout, leave the rest at their declared values
bazel test //... --test_timeout=^//integration:.*@600
# Stackable, like the existing flags
bazel test //... --test_timeout=^//slow:.*@1200 --test_timeout=^//flaky:.*@300
```
When no regex form is supplied, the flag would behave exactly as it does today (single value overrides all categories; four values override per category), so this is backward compatible.
### Which category does this issue belong to?
Core
### What underlying problem are you trying to solve with this feature?
A target can become timeout-flaky: intermittently exceeding its timeout under load or on a slower executor, or drifting upward in runtime until it routinely bumps against the declared limit.
The per-target levers that already exist — `--flaky_test_attempts` and `--runs_per_test` with the regex syntax — do apply here: a timed-out run is treated as a failure, so these flags retry timeouts. For a target that *occasionally* spikes (due to a race condition causing a hang, for example), that gives me an immediate, code-free, per-target mitigation, which is exactly what I want when operating CI: a devinfra team can take action to keep `main` green and prevent broken PRs without waiting on a code owner.
What's missing is a per-target way to *extend the timeout budget* rather than retry. Retrying is the wrong tool when a test has genuinely gotten slower rather than merely spiking:
1. **Ineffective** — if the test now consistently needs more time than the limit allows, every attempt times out and the target still ends up FAILED. Retries buy nothing.
2. **Expensive** — each attempt burns up to the full timeout before being killed. See #12314, where a 15m-timeout E2E target consumes an extra 15 minutes of a large worker per re-attempt for no benefit.
For that case the right fix is more headroom *per run* — a larger timeout — scoped to the affected target. Today that can only be done:
1. Globally / per-category via `--test_timeout` — too broad. It loosens timeouts for every test in the invocation (or every test in a size/timeout category), which hides genuinely-slow regressions elsewhere and weakens the signal the timeout exists to provide.
2. By editing the target's timeout or size attribute in `BUILD` — the change I'm specifically trying to avoid. It requires a source edit (and review/merge in a shared repo), and it permanently bakes a looser bound into the target rather than letting me apply a scoped, temporary override while I get the code owner to investigate.
3. `.bazelrc` — still global/per-category, same limitation as (1).
So the gap is a missing cell in the matrix: per-target control exists for retries/runs (`--flaky_test_attempts`, `--runs_per_test`) but not for the timeout budget (`--test_timeout`). Retrying — the only per-target lever available today — is ineffective and costly for a target whose runtime has actually increased. Bringing `--test_timeout` to parity would let teams extend the budget for a single drifting target the same way they already deflake a fail-flaky one.
### Which operating system are you running Bazel on?
_No response_
### What is the output of `bazel info release`?
_No response_
### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.
_No response_
### What's the output of `git remote get-url origin; git rev-parse HEAD` ?
```text
```
### Have you found anything relevant by searching the web?
#12314 ([FR] Fail fast and don't retry only in case of action timeout) — shows the community already treats retry-on-timeout as a distinct, costly behavior users want finer control over. A per-target timeout knob is the complementary lever: extend the budget instead of paying for repeated full-length retries.
#21142 (Custom RBE timeout for non-test actions) — closest in spirit; notes that test action timeouts can be raised via `--test_timeout` but there's no comparable per-action knob, and works around it via `execution_requirements`.
#19447 (@ character in label breaks `--flaky_test_attempts` parsing) — directly relevant to implementation: the `regex_filter@value` delimiter collides with @ characters that are legal in labels. Any per-target `--test_timeout` syntax would inherit the same parsing edge case and should be designed with #19447's resolution in mind.
#12411 (`--test_timeout` ignored by timeout warnings), #5879, #5015, #1748 — adjacent reports about timeout warnings misfiring; not the same request, but they show `--test_timeout` already interacts awkwardly with the size/timeout machinery.
### Any other information, logs, or outputs that you want to share?
_No response_
Contributor guide
Research direction
Start by reading the existing command-line handling for --runs_per_test and --flaky_test_attempts, then trace how --test_timeout parses its single-value and four-value forms. Done means repeated regex_filter@value overrides work for selected targets while preserving existing behavior, with coverage for parsing and the @ character edge case described in #19447.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- build-system, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100