DHI / DHI/mikecore-python

test_eum.py fails on master: eumItem is missing 110321, and CI pins an older native version that hides it

Open
#52 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5
Forks
1
PR merge metrics
No merged PRs in 30d

Description

`tests/test_eum.py::TestEUM::test_wrapper` fails on `master` today:

```
ValueError: 110321 is not a valid eumItem
```

It fails on the first assertion, at `tests/test_eum.py:8`. `eumWrapper.CreateItemHashtable()` asks the native EUM library for every item it knows and converts each id through `eumItem(...)`. The native library reports item `110321`, `"Carbon Emmision Factor"`. The Python `eumItem` enum in `mikecore/eum.py` ends at `eumIThickness = 110320`, so the conversion raises and the whole test dies before reaching anything it meant to check.

## Why CI does not see it

The workflows pin different versions of the same native package:

- `.github/workflows/test_linux.yml:31` — `nuget install DHI.MikeCore.Linux.rhel7 -Version 20.0.0`
- `.github/workflows/build_linux.yml:31` — `nuget install DHI.MikeCore.Linux.ubuntu -Version 22.1.0`

So the tests run against a native library old enough not to know `110321`, while the wheel that ships is built against one that does. Green CI, red suite for anyone installing the released package. Aligning the two versions would make CI reproduce this immediately.

## The underlying shape of the problem

`mikecore/eum.py` carries a hand-transcribed copy of the EUM item and unit tables — 607 `eumItem` members — with the maintenance burden stated in the source itself, twice (lines 9 and 611):

```python
# Must be updated with every new release, or if the EUM.xml is updated
```

Every binding to this library repeats that transcription and then drifts on its own schedule, so this failure is not a one-off missing constant — it is the mode this design fails in. Any new item added on the native side breaks `CreateItemHashtable()` in any binding that has not been hand-updated, and the break is total rather than per-item, because one unmappable id takes down the whole table.

## Two fixes, and they are independent

**Immediate:** add the missing member so the current native library round-trips:

```python
eumICarbonEmmisionFactor = 110321
```

(spelling copied from the native key, which has the typo)

**Structural:** stop the next one from being a test failure at all.

The root need is a **canonical, persistent, machine-readable source for EUM codes** — a stable public URL listing every item and unit id with its key — that bindings can generate from and that users can look a code up in. Today there is no such page: [MIKE Core SDK / EUM](https://docs.mikepoweredbydhi.com/core_libraries/core-libraries/) describes what EUM is, and [mikeio's EUM guide](https://dhi.github.io/mikeio/user-guide/eum.html) shows how to search types from Python, but neither enumerates the codes. The only complete list is the one each binding transcribes by hand, which is why they drift.

The model to copy is the [CF standard name table](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html): a numbered version, a persistent URL per version, a machine-readable [XML artifact](https://cfconventions.org/Data/cf-standard-names/current/src/cf-standard-name-table.xml) alongside the human-readable page, and a documented process for adding a name. Every client generates from it instead of transcribing it, and a citation like "CF standard names v88" is unambiguous years later. EUM has the same shape of problem — a controlled vocabulary of quantities and units consumed by many independent clients — and none of that infrastructure.

With that in place, `eum.py` becomes generated from a versioned artifact rather than maintained by hand, and a native version bump regenerates instead of breaking.

Until then, a smaller mitigation is worth having regardless: make id-to-enum conversion tolerant, so an id the enum does not know yields an unknown/passthrough item instead of raising and taking the whole hashtable down with it. One unrecognised code should not cost every other code in the table.

The immediate fix unblocks the suite. The canonical source is what stops this recurring on every release.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with tests/test_eum.py::TestEUM::test_wrapper and mikecore/eum.py, then compare the native package versions pinned in .github/workflows/test_linux.yml and build_linux.yml. The immediate result should cover EUM item 110321 and reproduce the suite against the shipped native version; the broader work requires a versioned machine-readable EUM source, generated bindings, and tolerant handling of unknown ids.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, devtools
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.