Zero culling forces model rebuilding
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 257
- Forks
- 87
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 29
Description
Version Checks (indicate both or one)
-
I have confirmed this bug exists on the lastest release of Linopy.
-
I have confirmed this bug exists on the current
masterbranch of Linopy.
Issue Description
When a constraint's LHS contains a term whose coefficient is exactly zero, linopy culls that term from the exported matrix, so the constraint's sparsity is data-dependent even though its feasible set is not.
Concretely, for the (common saturation-style) constraint
activation <= ratio * reservation ratio in [0, 1]
when ratio == 0 the term 0 * reservation is dropped, so the reservation column disappears from that row. Rebuilding the same model with ratio flipped between 0 and a non-zero value produces different LHS sparsity for the very same constraint.
The persistent / warm-start update machinery then cannot update the model in place: persistent/diff.py (diff_con, lines ~454-457) treats a change in the CSR indptr/indices as RebuildReason.SPARSITY, and Solver falls back to a full model rebuild + fresh solve (_rebuild). This defeats warm starts for any dynamic coefficient that crosses zero, which is a very common pattern (e.g. zero-price hours setting a bound to zero).
Root cause (likely introduced in #816) is the zero-coefficient culling at matrix export:
Constraint._matrix_export_data(linopy/constraints.py,valid_final = (vars_final != -1) & (coeffs_final != 0))Constraint.flat/to_polarsmask_func(mask = (data["vars"] != -1) & (data["coeffs"] != 0))
Reproducible Example
import numpy as np
import linopy
from linopy.persistent.diff import ModelDiff, RebuildReason
from linopy.persistent.snapshot import ModelSnapshot
from linopy.solvers import Solver
def build_model(ratios):
"""Two-timestep model: activation <= ratio * reservation."""
m = linopy.Model()
activation = m.add_variables(lower=0, name="activation", coords=[np.arange(2)])
reservation = m.add_variables(lower=0, name="reservation", coords=[np.arange(2)])
con = m.add_constraints(activation <= ratios * reservation, name="activation_limit")
m.add_objective(0 * activation) # feasibility problem
return m, con
def row_nnz(m, con):
csr, _, _, _ = con.to_matrix_with_rhs(m.variables.label_index)
return np.diff(csr.indptr).tolist()
# (1) Sparsity depends on the data: the zero coefficient is culled.
m1, c1 = build_model(np.array([1.0, 1.0])) # both rows: activation & reservation
m2, c2 = build_model(np.array([1.0, 0.0])) # row 2 coefficient == 0
print("ratio=[1, 1] nnz/row =", row_nnz(m1, c1)) # [2, 2]
print("ratio=[1, 0] nnz/row =", row_nnz(m2, c2)) # [2, 1] <- reservation col gone
# (2) The persistent update machinery sees a sparsity change and rebuilds.
snap = ModelSnapshot.capture(m1)
diff = ModelDiff.from_snapshot(snap, m2, same_model=False, ignore_dims=[])
assert diff is RebuildReason.SPARSITY
solver = Solver.from_name("highs", model=m1, io_api="direct", track_updates=True, set_names=False)
solver.solve(assign=True) # cold solve ok
solver.update(m2, ignore_dims=[]) # warm: only a coefficient changed to 0
assert solver._last_rebuild_reason is RebuildReason.SPARSITY
assert solver._rebuilds == 1 # full rebuild instead of in-place update
print("persistent solver rebuilt (RebuildReason.SPARSITY) instead of updating in place")
Output (verified on v0.9.1; the three assert lines pass, so rebuild_reason == SPARSITY and rebuilds == 1):
ratio=[1, 1] nnz/row = [2, 2]
ratio=[1, 0] nnz/row = [2, 1]
persistent solver rebuilt (RebuildReason.SPARSITY) instead of updating in place
Expected Behavior
Linopy should be smarter about the culling, or offer a way to opt-out of it where required
Installed Versions
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Constraint._matrix_export_data and the Constraint.flat/to_polars mask_func in linopy/constraints.py, then trace persistent/diff.py around lines 454-457 and Solver._rebuild. Run the reproducible example against the v0.9.1 behavior. Done means zero-valued coefficients no longer cause an unintended sparsity rebuild, or the supported opt-out behavior is covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100