Breakthrough-Energy / Breakthrough-Energy/PreREISE
Bug report: EIA API for demand data is timing out
- Dominant language
- Jupyter Notebook
- Stars
- 22
- Forks
- 29
- PR merge metrics
- No merged PRs in 30d
Description
# :beetle:
- [x] I have checked that this issue has not already been reported.
### Bug summary
The EIA API is proposed to be used for demand data for the HIFLD Project as described in Issue #293
I am trying to retrieve data for the list of BAs created in issue #241
During the week of Oct 31 - Nov 4, I have been attempting to use the EIA API, with varying degrees of success due to it timing out at unpredictable intervals, while trying to retrieve data for that BA list. When it times out, it throws an error. It is also slow, taking >10 minutes per BA in some cases.
### Code for reproduction
import getpass
import pandas as pd
import from prereise.gather.demanddata.eia.get_eia_data import get_ba_demand
start = pd.to_datetime('2016-01-01 00:00:00')
end = pd.to_datetime('2016-12-31 23:00:00')
key = getpass.getpass(prompt='api_key=')
#BA shp list is list of BAs from
get_ba_demand(ba_shplist, start, end, key)`
#
#
### Actual outcome

Other possible error I've observed: IncompleteRead: IncompleteRead(303104 bytes read)
#
#
### Expected outcome
What it looks like downloading data from the API:

### Additional context
There are multiple warnings that may be contributing to these issues that appear on the API site as of Nov 4, 2022.
There is a 'scheduled maintenance' on Nov 4, but the issue happened other days this week also.
There is a likely more important warning "Notice: EIA will discontinue support for its legacy API (APIv1) in November, 2022. Excel add-in v1 sheets will continue to function as they are. Please refer to our [documentation](https://www.eia.gov/opendata/documentation.php) for the APIv2 interaction methods and our [APIv2 query browser](https://www.eia.gov/opendata/browser/) to view the data."
This second warning may be contributing to the issues with the API and likely requires a long term fix.

Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.