NVIDIA / NVIDIA/cuopt

[BUG] Crossover fails with "A numerical error was encountered" on large LP (reproducible 2/2)

Open
#1,859 1 comment 0 reactions 1 assignee View on GitHub

@chris-maes is already working on this.

Since Sep 14, 2026.

Dominant language
Cuda
Stars
1k
Forks
233
Avg merge
4d 4h
Merged PRs (30d)
95

Description

Describe the bug

On a large LP (7.2M rows × 2.27M cols, 14.3M nnz), enabling crossover after a
method=1 (PDLP) solve fails reproducibly with a numerical error late in the
crossover phase:

  • Run 1: completes all 51,561 dual pushes, then aborts around push ~53.8k
    of 101,781 primal pushes, after 2 h 32 min total —
    Crossover status A numerical error was encountered., termination status 6,
    nan primal/dual objectives.
  • Run 2 (same structure, different variable bounds): completes all 44,975
    dual pushes, then the same error during the primal pushes, after 2 h 50 min.

The PDLP phase itself converges fine on both runs (relative gap ~1e-4 / 1e-5
in ~9 min on the GPU). The crossover phase runs single-threaded on CPU at
~100 % of one core with the GPU idle, which makes the failure expensive to
discover: ~2.5 h of compute before the nan.

Steps/Code to reproduce bug

settings = SolverSettings()
settings.set_parameter("time_limit", 14400)
settings.set_parameter("method", 1)        # PDLP
settings.set_parameter("crossover", 1)
solution = Solve(data_model, settings)     # LP: 7.2M rows x 2.27M cols, 14.3M nnz
# -> "Crossover status A numerical error was encountered.", status 6, nan objectives

Happy to share full solver logs. (The instance itself is large; I can try to
provide it or a reproducer on request.)

Expected behavior

Either a successful crossover to a basic solution, or a structured/early
failure — ideally with a partial-progress diagnostic (e.g. remaining primal
infeasibilities) instead of nan objectives after ~2.5 h of pushes.

Environment details

  • cuOpt version: 26.04.000 (git d9b7c96a)
  • Install method: Docker, nvidia/cuopt:latest-cuda12.9-py3.13
  • CUDA 12.9, driver-side GPU: NVIDIA GeForce RTX 3060 (12 GB)
  • CPU: AMD Ryzen 5 8500G (6C/12T), 31 GB RAM
  • OS: Ubuntu, kernel 6.8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.