EnergySystemsModellingLab / EnergySystemsModellingLab/MUSE_OS

[BUG] Global commodities column name mapping conflict

Open
#795 5 comments 0 reactions 2 assignees Claimed by @tsmbland View on GitHub
bug
Dominant language
Python
Stars
29
Forks
12
PR merge metrics
No merged PRs in 30d

Description

## Describe the bug

Older global_commodities input files will have headings Commodity, CommodityName etc.

The mapping from these old columns names to the new names is:

Commodity -> description
CommodityName - > commodity

There is a problem here: how does muse know whether a column called "commodity" is in the old parlance or the new parlance?

At present the mapping Commodity -> description does not seem to be listed in csv.py

If we added it, it would incorrectly map a *new style* column name, because the mapping is done after reducing to camel case.

One tempting fix would be to do the mapping before the case change. However, this would put us back into a situation of being case-sensitive, which we were trying to get away from...

Hopefully there is a better solution? (Or I have misunderstood the cause of the problem..?)

## To Reproduce

Run the function `standardize_dataframe()` from [csv.py](https://github.com/EnergySystemsModellingLab/MUSE_OS/blob/main/src/muse/readers/csv.py) on the input file [global_commodities.csv](https://github.com/EnergySystemsModellingLab/MUSE_UK_industries/blob/main/MUSEModel/global_input_files/global_commodities.csv)

## Expected behavior

`ValueError: Duplicate columns in Index(['commodity', 'commodity_type', 'commodity', 'emmission_factor'], dtype='object')'

## Context

Please, complete the following to better understand the system you are using to run MUSE.

- Operating system (eg. Windows 10): Mac OSX
- MUSE version (eg. 1.0.1): b779f2339abaddcc76e0b196192f8625ebb4486a
- Installation method (eg. pipx, pip, development mode): development mode
- Python version (you can get this running `python --version`): 3.9.18

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.