Sienna-Platform / Sienna-Platform/PowerOperationsModels.jl

CI Performance test:`HiGHS` solver pathologies

Open
#146 4 comments 0 reactions 1 assignee View on GitHub

@odow is already working on this.

Since Jul 22, 2026.

performance
Dominant language
Julia
Stars
2
Forks
1
Avg merge
1d 16h
Merged PRs (30d)
26

Description

The performance comparison shows weird behavior that can't be attributed to noise. Evidence:

  • Performance comparison on 2 stacked PRs showed 30% improvement in solve time against main, consistent across the two stacked PRs. Yet I confirmed that the PRs didn't change the objective function nor the number of constraints.
  • The next day, the same performance comparison runs for hours, both locally and on CI.

I had Claude code run the UC problem in the performance script, with the only difference being the HiGHS version and the threadcount. Results:

edit: misunderstanding in back-and-forth with claude code. Doesn't look like it's simply due to threadcount/version. Continuing to look into it. But the below items still seem like reasonable courses of action

Remedies:

  • Change the perf environment to use a fixed HiGHS minor version, instead of going with the most recent 1.0+ release
  • Separately record and report HiGHS solve time
  • Use single-threaded so it's deterministic
  • Increase MIP gap: under 1.23.0 multi-threaded, 1% (current) stalls but 2% solves quickly

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.