PyPSA / PyPSA/linopy

Add model diagnostics module

Open
#645 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

data-model enhancement performance
Dominant language
Python
Stars
257
Forks
87
Avg merge
1d 3h
Merged PRs (30d)
29

Description

Describe the feature you'd like to see

Add a model.diagnostics module/accessor that provides insights into model structure, quality, and performance characteristics. This module would be experimental — the API may change between releases as we learn what's most useful. Users will be warned accordingly.

Motivation

Users often build large models and have limited visibility into why solves are slow or produce unexpected results. A diagnostics module gives them tools to inspect and understand their models without needing deep solver expertise.

Ideas and use cases

Below is a non-exhaustive list of possible diagnostics. Not all need to be implemented at once — the module can grow over time.

Pre-solve (no solve needed, pure matrix/model analysis)
Diagnostic Use case
Scaling report — coefficient range ratios per row/column, flagging problematic ratios Identify numerical issues that slow down solvers or cause inaccurate results
Big-M detection — find constraints where binary variables appear with very large coefficients Spot big-M formulations that weaken LP relaxation and slow branch-and-bound
RHS/coefficient mismatch — flag constraints where RHS is on a very different scale than coefficients Catch common modeling mistakes that hurt solver performance
Model summary/profile — variable/constraint counts by type, sparsity, density of constraint matrix Quick overview to sanity-check model size and structure
Coefficient histogram/distribution — distribution of coefficient magnitudes across the model Understand overall scaling characteristics at a glance
Duplicate/near-duplicate constraint detection — find rows of A that are identical or nearly identical Catch redundant constraints that bloat the model
Empty/trivial constraint detection — constraints with no variables or that are trivially satisfied Catch modeling bugs early
Variable bound analysis — variables with very wide or very tight bounds, or missing bounds Spot variables that may need tighter bounds for performance
Post-solve (requires a solve)
Diagnostic Use case
Constraint activity — classify constraints as tight, slack, or violated; report slack values Understand which constraints are binding and which are redundant in the solution
Variables at bounds — which variables are at their lower/upper bound Identify bottlenecks in the solution
LP relaxation quality — solve LP relaxation, report integrality gap and fractionality per integer variable Assess MIP formulation strength and identify variables with weak relaxation
Dual value analysis — distribution and extreme values of dual variables Understand shadow prices and constraint sensitivity
Solver-dependent (requires specific solver backends)
Diagnostic Use case
IIS analysis (already exists via compute_infeasibilities) Diagnose infeasible models
Basis condition number Assess numerical stability of the current basis
Design considerations
  • Experimental API — this module should clearly warn users (e.g. via a FutureWarning on first use or in docs) that the API is not yet stable and may change
  • Accessor pattern — similar to model.matrices, accessed via model.diagnostics
  • Lazy computation — diagnostics should be computed on demand, not on model creation
  • Solver-agnostic where possible — most diagnostics should work on the model's xarray data structures without requiring a specific solver
  • Return xarray objects — results should use xarray DataArrays/Datasets to stay consistent with linopy's data model and preserve labeled dimensions. This makes it easy to cross-reference diagnostics with the original variables and constraints
  • Non-invasive — diagnostics should never modify the model (LP relaxation analysis should restore state via unrelax())
What this is NOT
  • Not an automatic reformulation tool — diagnostics identify issues, the user decides how to fix them
  • Not a replacement for solver-specific analysis tools — but a solver-agnostic starting point
  • Not a stable API commitment (initially) — we should be free to iterate on what's useful

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing model.matrices accessor and the model's xarray data structures. The issue lists possible pre-solve, post-solve, and solver-dependent diagnostics but does not define an initial subset or implementation files. Clarify the first diagnostic scope, experimental warning behavior, lazy computation, and xarray return shapes before estimating completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.