Text-based (YAML) math definition interface
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 257
- Forks
- 87
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 29
Description
It would be useful to be able to define all the mathematics of a linopy model in human-readable text format. This would help with transparency and shareability (if that's a word) of math, as well as making it easier to define new math without needing to be so familiar with Python.
I have developed something that I think would be suitable in the Calliope project. It is a vectorised syntax that is documented. The main ideas are:
- it works with xarray dataarrays, using a
foreachdirective to define the array dimensions and awheredirective to mask the array so that variables/exprs/constraints can be defined only for a subset. - Math expressions do not reference dimensions unless slicing specific elements in the math. That is, you could define
p <= p_nomand it would be applied automatically to all entries in the dimensions defined inforeach, or you could definep[snapshot=2012-01-01] <= p_nom * 0.8and it would slicepto a given snapshot. - helper functions exist to deal with more complex cases. An obvious use-case is to roll the timeseries dimension forward/backward on one element so that you can get the equivalent of
state_of_charge[t] = state_of_charge[t-1] + p_store[t] - p_discharge[t]by callingstate_of_charge = roll(state_of_charge, snapshot=1) + p_store - p_discharge. These are designed as plugins so that power users can add and use new ones if the need arises. - It can automatically be rendered as LaTeX math for use in documentation. This means you only document your math once, when defining it in YAML, and it can then be kept up-to-date in the docs automatically.
It may not be fit for all linopy purposes but the fact that it also relies on multi-dim xarrays I think is already a good starting point. I have been exploring how it could also be used for tidy dataframes (if e.g. updating to using pyoframe) and it seems relatively straightforward; I just need to better understand how to apply the where string masking on polars dfs.
To mitigate bad actors injecting code into the expressions, all strings are parsed with pyparsing (ref. where and expression strings).
Examples of use for user-defined math can be seen here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing Calliope's linked src/calliope/math/base.yaml, backend/where_parser.py, and backend/expression_parser.py, along with its user-defined-math documentation and examples. Compare that design with linopy's current model and expression interfaces, then define the supported YAML syntax, parsing boundaries, and rendering behavior before implementation; done means the scope and integration points are agreed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100