Unit incompatibility error when reporting some scenarios
- Dominant language
- Jupyter Notebook
- Stars
- 150
- Forks
- 178
- Avg merge
- 17h 32m
- Merged PRs (30d)
- 2
Description
Simple message_ix reporting gives the following error for some sets of models (e.g. ENGAGE, NGFS) but not for some other models (e.g. MESSAGEix-Materials). This error is coming from either `fom` or `vom` key, but those incompatible units are not found from corresponding parameters.
#### Code sample or context
```python
import ixmp as ix
import message_ix
from message_ix.reporting import Reporter
mp = ix.Platform('ixmp_dev')
Sc_ref = message_ix.Scenario(mp, 'ENGAGE_SSP2_v4.1.7', 'EN_NPi2020_1400')
rep = Reporter.from_scenario(Sc_ref)
df = rep.get("message:default")
```
#### Expected result
An `df `output of type `pyam.core.IamDataFrame`
#### Problem description
Error (Units incompatible)
```
(base) C:\WINDOWS\system32>mix-models --url="ixmp://ixmp_dev/ENGAGE_SSP2_v4.1.7/EN_NPi2020_1400" dle report --old_reporting False
C:\ProgramData\Anaconda3\lib\site-packages\sdmx\session.py:13: RuntimeWarning: optional dependency requests_cache is not installed; cache options to Session() have no effect
RuntimeWarning,
2021-06-28 11:24:05,172 INFO at.ac.iiasa.ixmp.Platform:146 - Welcome to the IX modeling platform!
2021-06-28 11:24:05,174 INFO at.ac.iiasa.ixmp.Platform:147 - connected to database 'jdbc:oracle:thin:@x8oda.iiasa.ac.at:1521/PIXMP2.iiasa.ac.at' (user: ixmp_dev)...
This Scenario has a solution, use `Scenario.remove_solution()` or `Scenario.clone(..., keep_solution=False)`
ixmp.model.base.initialize_items This Scenario has a solution, use `Scenario.remove_solution()` or `Scenario.clone(..., keep_solution=False)`
genno.config.units Replace unit '-' with ''
fix_cost: mixed units ['USD/GWa', 'USD/kWa'] discarded
ixmp.reporting.computations.data_for_quantity fix_cost: mixed units ['USD/GWa', 'USD/kWa'] discarded
inv_cost: mixed units ['USD/GWa', 'USD/kWa'] discarded
...data_for_quantity inv_cost: mixed units ['USD/GWa', 'USD/kWa'] discarded
emission_factor: mixed units ['tC', '???', 'kg/kWa', '-'] discarded
...data_for_quantity emission_factor: mixed units ['tC', '???', 'kg/kWa', '-'] discarded
input: mixed units ['GWa', '-', '???'] discarded
...data_for_quantity input: mixed units ['GWa', '-', '???'] discarded
output: mixed units ['GWa', '-'] discarded
...data_for_quantity output: mixed units ['GWa', '-'] discarded
numexpr.utils._init_num_threads NumExpr defaulting to 8 threads.
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\Scripts\mix-models-script.py", line 33, in
sys.exit(load_entry_point('message-ix-models', 'console_scripts', 'mix-models')())
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\click\decorators.py", line 33, in new_func
return f(get_current_context().obj, *args, **kwargs)
File "h:\mydocuments\message\message_data\message_data\model\dle\__init__.py", line 168, in run_reporting
report(scenario, old_reporting)
File "h:\mydocuments\message\message_data\message_data\model\dle\reporting.py", line 192, in report
df = rep.get("message:default")
File "C:\ProgramData\Anaconda3\lib\site-packages\genno\core\computer.py", line 407, in get
raise ComputationError(exc) from None
genno.core.exceptions.ComputationError: computing using:
(, 'fom:nl-t-yv-ya', 'vom:nl-t-yv-ya')
Use Computer.describe(...) to trace the computation.
Computation traceback:
File "C:\ProgramData\Anaconda3\lib\site-packages\genno\computations.py", line 74, in add
raise ValueError(f"Units '{ref_unit:~}' and '{u:~}' are incompatible")
ValueError: Units '' and 'USD / GWa' are incompatible
```
#### Versions
Output of message-ix show-versions
```
ixmp: 3.3.0a0
0188de0 (HEAD -> origin/main) Re-enable py3.9 runs of "pytest" CI workflow
message_ix: 3.2.1.dev55+g2107fbc.d20210506
a73cc36 (HEAD -> master, VK/master) Merge pull request #456 from iiasa/issue_437
message_data: 0.1.dev0
6c656e01 (HEAD -> DLE-MESSAGE, Jihoon/DLE-MESSAGE) Add an emission constraint
click: 7.1.2
dask: 2021.04.0
graphviz: 0.12
jpype: 1.2.1
… JVM path: C:\Program Files\Java\jdk1.8.0_202\jre\bin\server\jvm.dll
openpyxl: 3.0.7
pandas: 1.2.4
pint: 0.14
xarray: 0.17.0
yaml: 5.4.1
iam_units: installed
jupyter: installed
matplotlib: 3.3.4
plotnine: 0.7.1
pyam: 0.10.0
GAMS: 26.1.0
python: 3.7.10 (default, Feb 26 2021, 13:06:18) [MSC v.1916 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None
```
Contributor guide
Research direction
Reproduce the ENGAGE example with Reporter.from_scenario and rep.get("message:default"). Start in message_data/model/dle/reporting.py at report(), then trace the genno computation for fom and vom and inspect the reported parameter units. Done means the example returns an IamDataFrame without the incompatible-unit error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100