[BUG] Flaky test: c_api/TimeLimitTestFixture.time_limit/2 (MIP) overshoots time budget on constrained CI runners
@Kh4ster is already working on this.
Since Apr 22, 2026.
- Dominant language
- Cuda
- Stars
- 1k
- Forks
- 233
- Avg merge
- 4d 4h
- Merged PRs (30d)
- 95
Description
Summary
c_api/TimeLimitTestFixture.time_limit/2 (the MIP parametric case using /mip/supportcase22.mps, time_limit=15s, CUOPT_METHOD_DUAL_SIMPLEX) fails intermittently on CI when the reported solve_time exceeds target_solve_time + excess_allowed_time (15s + 3s = 18s).
Failing assertion
cpp/tests/linear_programming/c_api_tests/c_api_tests.cpp:64
EXPECT_NEAR(solve_time, target_solve_time, excess_allowed_time); // excess = 3.0
Observed CI failure
solve_time = 18.398s, target15s, tolerance3.0s→ over by ~0.4s.- Solver itself respected the limit: log shows
Explored 0 nodes in 15.01s.and termination statusTime limit (5). - CI environment per log: AMD EPYC 9554, only 12/12 threads visible (cgroup-limited on a 64-core host), 43 GiB RAM, NVIDIA H100 NVL, CUDA 13.1.
Root cause
Test reads cuOptGetSolveTime, which for MIP returns mip_solution_t::get_total_solve_time() → stats_.total_solve_time. That field is recorded at cpp/src/mip_heuristics/solver.cu:489 (and similar check_time_limit guards) using a timer_t that was started in cpp/src/mip_heuristics/solve.cu:306 — before Papilo presolve.
Therefore solve_time includes:
- Papilo presolve (OpenMP, ~2.59s in this run)
- cuOpt presolve (~1.59s)
run_mip(LP root relaxation, primal heuristics, B&B)- Post-B&B serial wind-down inside
mip_solver_t::solve():branch_and_bound_status_future.get(),sol.compute_feasibility(),sol.test_variable_bounds(...)+ device sync
It does NOT include Papilo postsolve, the full_sol rebuild, or C-API wrapping (those run after total_solve_time is finalized).
The 3.4s gap between B&B's internal stop (15.01s) and total_solve_time (18.40s) is the post-join serial wind-down. On CPU-thread-constrained runners:
- Papilo presolve takes a larger share of the 15s budget (it's OpenMP-parallel — see
cpp/src/mip_heuristics/solve.cu:412passingsettings.num_cpu_threads). - B&B uses
omp_get_max_threads() - 1worker threads (cpp/src/mip_heuristics/solver.cu:360). - The post-B&B
compute_feasibilityover the ~260K-row presolved problem becomes proportionally more expensive vs the budget.
Memory is not a factor at this problem size (~260K rows / 2.2M nz, peak well under 1 GiB).
Reproduction
- CI: failure observed on H100 NVL CI runner with cgroup-limited 12-thread visibility.
- Local (RTX 8000, compute 7.5): not reproducible — full
c_api/TimeLimitTestFixture.time_limit/*set passes.
Suggested fixes (ranked)
- Measure
solve_timefrom inside the time-limited region only so the field reflects whattime_limitactually controls. Excludes presolve/post-solve serial work that the user'stime_limitsetting cannot bound. - Per-case
excess_allowed_time— keep 3s for LP cases, raise (e.g. 7s) for the MIP case to absorb host-side wind-down variance. - Increase
target_solve_timefor the MIP case so post-solve overhead becomes a smaller % of the budget.
(1) is the principled fix; (2)/(3) are pragmatic unblocks.
Mitigation in progress
Disabling the /2 case in a follow-up PR while this is investigated; the LP /0 and /1 cases remain enabled.
Related
There is a stale TODO at cpp/src/dual_simplex/presolve.cpp:1517 mentioning this test name, but it is about a different (numerical assertion) failure mode, not this timing one.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.