gridstatus / gridstatus/gridstatus
Improving status info in ISONE, SPP, & ERCOT interconnection queues
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 445
- Forks
- 87
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 8
Description
Hello 👋🏼I work for Catalyst Cooperative and am starting to use GridStatus to pull ISO interconnection queues. Thanks so much for your work building this functionality and publishing it as open source!
While exploring the queue data, I ran into a couple of minor issues. I'm happy to submit a PR in the next week or two to address them, but wanted to outline them here in case somebody else is already working on any of them.
- In SPP, GridStatus currently overwrites more detailed status information with simplified categories. Simple categories are useful, but preserving the detailed information is also useful for tracking project progress. I think this is as simple as making a new column to preserve the original values.
- ISONE publishes their interconnection queue as an HTML table (great!) but they encode queue status information as graphical icons (less great), with the category label as the title text in each
<img>tag. GridStatus does not currently parse this information, so it is all lost. It should be pretty easy to fix by, for example, replacing the<img>tags with their title text prior topd.read_html(). - Also in ISONE, GridStatus defines
Queue IDwith what the original data callsQueue Position. Queue Position is only unique for active projects. It has a 1:m relationship to both withdrawn and completed projects. IMHO considering theQueue IDis unique in all the other ISOs (excepting a tiny number of errors in the source data), it is a bit sacrilegious to label a non-unique value with the sacred letters "ID". Unfortunately I don't see a natural key for those items -- perhaps we can come up with a surrogate key? - I think you already call this out in #76, but to be more explicit: when comparing ISOs, status category labels that are superficially synonymous can actually differ considerably in their meaning. For example, in MISO & PJM queues, the term "active" refers to projects up to the point of IA execution, whereas in CAISO, ERCOT, ISONE, & NYISO queues "active" applies up to the point of commercial operation, and the sickos at SPP call every project that wasn't withdrawn "active", including operational facilities. There is often a year+ between IA execution and commercial operation, during which construction, energization, and finally synchronization occur. I think it would be useful for GridStatus to define consistent boundaries across ISOs, however arbitrary they may be. But the raw data should also be available to enable alternative interpretations.
- Related to the above (and probably subordinate to it), in ERCOT, GridStatus defines "active" and "completed" using
queue["IA Signed"].isna(). While in line with some ISO's taxonomies, I think that is actually inconsistent with ERCOT's definition. ERCOT has a separate sheet for "inactive" projects (not processed by GridStatus), from which I infer that the projects processed by GridStatus are "active".
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked sections of gridstatus/spp.py, gridstatus/isone.py, and gridstatus/ercot.py, then review issue #76 for related status semantics. Clarify which raw values, identifiers, and cross-ISO status boundaries should be supported before changing the queue parsing. Done means the agreed status information is preserved or normalized consistently across the affected ISO datasets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100