QuantEcon / QuantEcon/lecture-python-programming
wb_download.py: use HTTPS, add a timeout and raise_for_status()
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 72
- Forks
- 31
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 8
Description
lectures/_static/lecture_specific/pandas/wb_download.py fetches the World Bank indicator over plain HTTP and writes the response body to disk without checking whether the request succeeded. Raised by Copilot while reviewing the downstream port at QuantEcon/lecture-python-programming.ml#6; it is upstream code, so it belongs here rather than in a translation edition.
Current lines 6-9:
wb_data_query = "http://api.worldbank.org/v2/en/indicator/gc.dod.totl.gd.zs?downloadformat=excel"
r = requests.get(wb_data_query)
with open('gd.xls', 'wb') as output:
output.write(r.content)
Three points, in rough order of how much they matter:
No error check. On any non-200 response the error page body is written to gd.xls, and the failure only surfaces further down as an opaque pd.read_excel parse error rather than as the network problem it actually is. r.raise_for_status() turns that into a clear failure at the point of the fault.
Plain HTTP. api.worldbank.org serves HTTPS, so this is a free upgrade — and it is a file the lectures hold up as example code, which is the argument for getting it right beyond the MITM exposure itself.
No timeout. requests.get without timeout blocks indefinitely if the endpoint stops responding, which is an unpleasant failure mode inside a notebook build.
Suggested:
wb_data_query = "https://api.worldbank.org/v2/en/indicator/gc.dod.totl.gd.zs?downloadformat=excel"
r = requests.get(wb_data_query, timeout=30)
r.raise_for_status()
with open('gd.xls', 'wb') as output:
output.write(r.content)
Note this script is a static asset rather than an executed cell, so nothing in CI exercises it — worth a manual check that the HTTPS endpoint returns the same workbook before merging.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open lectures/_static/lecture_specific/pandas/wb_download.py and inspect the fetch at lines 6-9. Update the request as described, then manually check that the HTTPS World Bank endpoint returns the expected workbook, since this static asset is not exercised by CI. Done means failures surface clearly and the workbook still supports the downstream read.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100