PyPSA / PyPSA/pypsa-usa

Refactor `build_demand.py` to use EIA930 data from PUDL

Open
#313 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
148
Forks
55
Avg merge
2d 11h
Merged PRs (30d)
18

Description

Feature Request

Catalyst Coop is currently in the midst of adding EIA930 data to PUDL.

They just produced their first nightly build that includes (relatively unprocessed) hourly EIA-930 balancing authority operations data, including the following 4 tables:

  • core_eia930__hourly_interchange
  • core_eia930__hourly_net_generation_by_energy_source
  • core_eia930__hourly_operations
  • core_eia930__hourly_subregion_demand

Docs are here, notebook poking at the data here.

Once the tables graduate to out_eia930_*, meaning they have been fully cleaned up and are production ready, we'll need to refactor build_demand to pull EIA930 data from PUDL.

Suggested Solution
  • Identify which PUDL tables / columns we need
  • Check in with Zane from Catalyst about when out versions of the data will be out
  • Refactor workflow/scripts/building_demand.py to query tables for data
  • Update Snakefile to make build_demand task depend on forthcoming retrieve_pudl task (#311)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the linked PUDL EIA930 documentation and sanity-check notebook, then inspect workflow/scripts/building_demand.py and the Snakefile. Identify the required PUDL tables and columns, confirm that the out_eia930 data is production-ready, and trace the dependency on the forthcoming retrieve_pudl task. Done means build_demand uses the cleaned PUDL data and the workflow dependency is updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, data-engineering
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.