Breakthrough-Energy / Breakthrough-Energy/PreREISE

Add more flexibility for loading EIA/EPA data when building HIFLD grid

Open
#246 0 comments 0 reactions 4 assignees Claimed by @YifanLi86 View on GitHub
feature request hifld
Dominant language
Jupyter Notebook
Stars
22
Forks
29
PR merge metrics
No merged PRs in 30d

Description

# :rocket:

- [ ] Is your feature request essential for your project?

### Describe the workflow you want to enable
Currently, we're loading EIA & EPA data for a specific year from CSVs on our blob storage, which were either downloaded as zip files (EPA AMPD) or created manually from xlsx files (EIA Form 860). I with that there were more user-accessible flexibility, in terms of being able to download data for a given year (or a given month, since EIA has monthly Form 860M releases inbetween the annual Form 860 release), and in terms of being able to obtain data from different sources (e.g. from a local copy of Catalyst Cooperative's PUDL database, or their web API).

### Describe your proposed implementation
Additional functions could be added to `prereise.gather.griddata.hifld.data_access.load` which could read data from different sources, and additional parameters could be added to the highest-level functions within `prereise.gather.griddata.hifld.data_process` to specify which data sources to read from at the start of processing, and these same parameters could be added to `prereise.gather.griddata.hifld.orchestration.create_csvs` and then passed through to the `data_process` functions.

### Additional context
Catalyst Cooperative currently has a subset of the data we need available via a Datasette interface (which can be read directly from pandas), but not the full dataset, which is available as a sqlite database.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.