[BUG] ROUTING_TEST CVRPTW_Retail/4 SIGABRT — "Cost should improve" assertion in execute_vrp_moves
@aliceb-nv is already working on this.
Since May 21, 2026.
- Dominant language
- Cuda
- Stars
- 1k
- Forks
- 233
- Avg merge
- 4d 4h
- Merged PRs (30d)
- 95
Description
Describe the bug
ROUTING_TEST crashed with SIGABRT on the level0_retail/retail_float_test_t.CVRPTW_Retail/4 case. The cuopt_assert invariant on the aggregate cost change after a batch of VRP local-search moves did not hold:
ROUTING_TEST: cpp/src/routing/local_search/vrp/vrp_execute.cu:466:
bool cuopt::routing::detail::execute_vrp_moves(
solution_t<i_t, f_t, REQUEST>&, move_candidates_t<i_t, f_t>&, i_t)
[with i_t = int; f_t = float;
cuopt::routing::request_t REQUEST = cuopt::routing::request_t::VRP]:
Assertion `cost_before - cost_after > EPSILON && "Cost should improve!"' failed.
./ci/run_ctests.sh: line 86: 4820 Aborted (core dumped) "${gt}" --gtest_output="xml:${xml_file}" "$@"
CRASH: ROUTING_TEST died from SIGABRT (exit code 134)
The assertion fires after a batch of accepted VRP moves is executed: the planner predicted a cost decrease, but the post-execution measured cost decrease was not greater than EPSILON. This indicates either stale move-candidate data, conflicting moves applied together, or a predicted/actual delta mismatch.
Where
- File:
cpp/src/routing/local_search/vrp/vrp_execute.cu:466 - Function:
cuopt::routing::detail::execute_vrp_moves<int, float, VRP> - Test:
level0_retail/retail_float_test_t.CVRPTW_Retail/4 - Binary:
ROUTING_TEST(debug build — assertion is insidecuopt_func_call/cuopt_assert)
Reproducer / observed run
- PR: #1269 (CI-only change — does not touch routing code)
- Run: actions/runs/26237894930
- Job:
conda-cpp-tests / 13.2.0, 3.13, amd64, ubuntu24.04, rtxpro6000, latest-driver, latest-deps(job 77225276444) - Head SHA:
371edafa(branchpull-request/1269, baserelease/26.06) - Surrounding tests
CVRPTW_Retail/0..3passed; only/4aborted.
Expected behavior
execute_vrp_moves should only accept and execute moves whose net post-execution cost change matches the predicted delta within EPSILON. The assertion should hold for every batch on every accepted test instance.
Environment
- CI runner:
rtxpro6000(NVIDIA RTX PRO 6000 Blackwell Server Edition, 94.97 GiB VRAM) - CPU: INTEL(R) XEON(R) GOLD 5512U, 16 threads
- CUDA: 13.2
- Driver: latest-driver
- Deps: latest-deps
- Build: conda, ubuntu24.04, amd64, Python 3.13
- Branch under test:
release/26.06
Related issues
- #1221 (closed 2026-05-18) — similar
CVRPTW_RetailSIGABRT, but on case/18and root-caused to route-priority-sort out-of-bounds (fixed by 1d85aed6). Current failure is on/4and at a different assertion site, so likely a distinct root cause that survived that fix. - #866 (closed 2026-05-18) — same family of cost-invariant assertion but at
local_search.cu:315("Cost mismatch after a move"), case/18. Cross-link for context. - #402 (closed 2025-11-20) — older instance of the same "Cost should improve" assertion family.
Additional context
- PR #1269 modifies only
.github/workflows/*andci/release/update-version.sh, so the failure cannot have been caused by its changes. Either:- The test is flaky on this hardware/CUDA combination, or
- A latent bug exists on
release/26.06that this run happened to expose.
- If flaky: consider whether
cuopt_assertshould be relaxed for the aggregated-batch case, or the failing instance saved for offline reproduction. - If a real regression: bisect
release/26.06since #1221's fix landed (1d85aed6 on 2026-05-18) to identify the change that exposed it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.