Preprocessing script (fixing inconsistent DEA entries)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 130
- Forks
- 59
- PR merge metrics
- No merged PRs in 30d
Description
I'm becoming a bit frustrated by the inconsistent data format of the DEA input data.
E.g.: Column orders and names being inconsistent, most unchanging values are filled across all years while sometimes they are not. The inconsistencies for "Hydrogen to Jet Fuel" are e.g. at an extend were I simply started extracting the numbers by hand rather than automatically =/
The inconsistencies also make the compile_cost_assumption.py script more and more bloated and unreadable.
Suggestion:
Have a pre-processing script which is responsible for creating a consistent input data format and move all "if this tech than special treatment" cases there.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with compile_cost_assumption.py and inspect how DEA inputs are currently parsed, especially the Hydrogen to Jet Fuel data. Define a consistent preprocessed input format and move the technology-specific handling there; done means the compilation script is less bloated and can consume the normalized data consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100