MACRO-calibration incomplete cleanup for set("cat_node")
- Dominant language
- Jupyter Notebook
- Stars
- 150
- Forks
- 178
- Avg merge
- 17h 32m
- Merged PRs (30d)
- 2
Description
When applying the function `add_macro` to a scenario, the set `cat_node` is not removed, prior to being re-populated.
The issue in more detail:
- An `R11` model version, which has already been calibrated to macro, is split to a new region version `R17`.
- The regions `CPA`, `FSU`, `PAO`, `PAS` have been split into new regions.
- The 4 above regions remain in the scenario as these still have the land-use emulator implemented, as GLOBIOM doesnt have all the new regions. The regions have NO energy-system elements and therefore no demands; therefore they also DO NOT need to be calibrated to macro.
- The macro-input excel file has NO mention of the 4 regions listed above; they are however still part of the set `cat_node`.
- Calling the calibration process via `scen.add_macro()` results in `UNDF` values for the 4 regions, e.g. for `demand_scale` above and calibration fails. This will be logged in the file `MACRO_run.lst`.
How to resolve:
- A current work around is to remove the set `cat_node` prior to sunning the macro calibration process.
```
with scen.transact("Remove set cat_node"):
df = scen.set("cat_node", filters={"type_node": "economy"})
scen.remove_set("cat_node", df)
```
A possible fix can be added [here ](https://github.com/iiasa/message_ix/blob/bf4469111a2d10c5dbc2c921e6a7e502e96aea34/message_ix/macro.py#L194) in the calibration process
#### Versions
```
ixmp: 3.6.1.dev38+g43562c1
43562c1 (HEAD, upstream/main) Merge pull request #466 from iiasa/reuse-workflows
message_ix: 3.6.1.dev49+g3b6651f.d20221205
7d8ac40 (HEAD, upstream/issue/723, issue/723) Revise duality test setup
```
Contributor guide
Research direction
Start in macro.py around line 194 and trace the add_macro calibration process. Reproduce the case with the split regions and inspect MACRO_run.lst for the UNDF values; compare with the documented cat_node removal workaround. Done means calibration no longer retains the excluded regions or fails with UNDF values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100