PyPSA / PyPSA/linopy

Math-spec integration

Open
#919 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement math-spec
Dominant language
Python
Stars
257
Forks
87
Avg merge
1d 3h
Merged PRs (30d)
29

Description

While the math-spec package (https://github.com/energy-models/math-spec/) is actively evolving, ground work for integrating it into linopy and shaping the API surface can already be tackled. The following proposal comes with couple of logics allow to define 1) a whole optimization problem, 2) parts of math through the spec. In 3) I envisioned a logic that uses the expressions from math-spec and reports their values.

  1. a function that defines a linopy model from the spec:
class Model:
    ...
    @classmethod
    def from_spec(cls, spec, sources: Mapping[str, Any]) -> Model:
    ...

It takes spec as math-spec-compatible, ie. anything that math_spec.to_program accepts (YAML path, YAML string, dict, Spec, Program) and input data sources as a Mapping. The explicit format of the latter is to be discussed. I could be compatible with long form tables as from datarecord or even work model.parameters. The function returns a linopy Model instance.

  1. a function that incrementally updates a model with a spec
class Model:
    ...
    def add_spec(self, spec: mathspec.Buildable, sources: Mapping[str, Any] | None) -> None:
    ...

Same inputs, but now extending an already initialized model. Likely, sources could be optional as math could run on existing variables.

3a) a function to report numeric results of expressions that are defined by the spec. When math-spec defines expression may they be data-only, constraint expressions or post-solve expressions (see https://github.com/energy-models/math-spec/pull/287), linopy does explicitly not load them into memory but keeps them in the spec layer, the same as lpspec does today. Such a setup follows the lazy spirit from #888, but now with the math+data as the only reference. It returns a dataarray.

class Model:
    ...
    def report(self, str, sources: Mapping[str, Any] | None) -> xarray.DataArray:
    ...

3b) alternatively a Report class that is attached to the linopy model and points to all math expressions from the spec as well as in-memory expressions contained by the model. you can materialize the report by selecting an item.

linopy.model.report["lcoe"] -> xr.DataArray

While I personally like such an accessor more than the function, here the question arises how to keep the input data tight to the model and make references to the data work out. Therefore you would likely want to initialize the Report class with the sources as argument.

From an API perspective, you could argue, why does linopy need to care about post-solving and data analysis as proposed by 3a+b). math-spec is targeting such a capability for good reasons, and I believe we should leverage them here. Where I am not sure is how to organize the implementation strategically.
While I implemented the expression calculation in the linopy lane here https://github.com/fluxopt/lpspec/pull/1518; I start to question the road. A better way could be to only implement the non-optimization expressions through the relational lane and convert the result to xarray and reporting through linopy's Report class. However this would tie us permanently to lpspec which I would be fine with, but I would like to discuss that point.

tagging @brynpickering @FBumann @coroa

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the proposed Model.from_spec, add_spec, and report APIs, then review the linked lpspec pull request and the math-spec expression behavior. The work is done only after the integration approach and data-source handling are agreed and the selected API supports model construction, incremental updates, and expression reporting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.