boostorg / boostorg/unordered

Too many (separate) tests?

Open
#120 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
92
Forks
88
PR merge metrics
No merged PRs in 30d

Description

There may be to many individual test binaries which slow down the build/CI. See e.g. https://github.com/boostorg/unordered/runs/6779803837?check_suite_focus=true

For `b2 -j3 libs/%LIBRARY%/test toolset=msvc-14.3 cxxstd=14,17,20,latest address-model=32,64 variant=debug,release embed-manifest-via=linker` it already yields (at least) 4*2*2=16 variants of each test.

In the test Jamfile we have 54 run-tests and 4 compile-tests yielding at least 928 compilations. That job already takes over an hour on GHA. With coverage collection it is even over 3h and counting.

And it seems the calculation is still to low: `...updated 7154 targets...` O.o

Is it possible to trim that down? E.g. combine more tests into single compilation units as especially on Windows firing up the compiler/linker is very expensive (in terms of time).

Also I'm not sure where the additional factor of 7 (in the targets) is coming from. Maybe that could also be trimmed down somehow.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the test Jamfile and the supplied b2 command, then inspect the linked GitHub Actions run to compare test variants, compilations, and reported targets. Determine where the extra targets come from and identify which test binaries or variants can be combined without losing coverage; done means a reduced build workload with the existing tests still running.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system, performance, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.