bazelbuild / bazelbuild/bazel

Set default test size to "small"

Open
#22,355 3 comments 9 reactions 0 assignees View on GitHub
team-Bazel type: feature request untriaged
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 20h
Merged PRs (30d)
72

Description

### Description of the feature request:

If unspecified, a test's size currently defaults to "medium". I think small is a far more reasonable default.

[aspect_bazel_lib](https://github.com/aspect-build/bazel-lib/blob/1697a3275becc29d0c002f6ef18bd63885b4d758/lib/private/utils.bzl#L139-L171) says:

```
The [test-encyclopedia](https://bazel.build/reference/test-encyclopedia) says:
* Tests may return arbitrarily fast regardless of timeout.
* A test is not penalized for an overgenerous timeout, although a warning may be issued:
* You should generally set your timeout as tight as you can without incurring any flakiness.

However Bazel's default for timeout is medium, which is dumb given this guidance.
```

I'm inclined to agree with this. The *vast* majority of tests should be marked as small, so it would be more convenient, and given the guidance, it would make sense to set the timeout as tight as is allowed (small) by default, and then bazel will timeout and tell you to increase the timeout.

To achieve this, I request creating a flag `--incompatible_default_test_size` and default it to medium. Then in a future release, we can change it to small, before eventually deleting the flag.

### Which category does this issue belong to?

Java Rules

### What underlying problem are you trying to solve with this feature?

Our pre-upload checks add the flag `--test_size_filters=small`. I just recently ran into an issue where the size of a test was not specified, and so the test was skipped. I think defaulting it to small is far more useful, and also means that users don't have to remember to set the test size - instead, they will know to set the test size because the test fails from timing out.

### Which operating system are you running Bazel on?

Linux

### What is the output of `bazel info release`?

7.1.1

### If `bazel info release` returns `development version` or `(@non-git)`, tell us how you built Bazel.

_No response_

### What's the output of `git remote get-url origin; git rev-parse HEAD` ?

_No response_

### Have you found anything relevant by searching the web?

_No response_

### Any other information, logs, or outputs that you want to share?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing Bazel's handling of an unspecified test size and the proposed --incompatible_default_test_size flag, then review how --test_size_filters=small interacts with the current medium default. Done means the flag can preserve the existing behavior while providing a path to a future small default, with coverage for both settings.

Written by the indexing model from the issue text.

Assessment

Tech stack
build-system
Domain
build-system, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.