Percentile Calculator offers stations it has no data for
- Dominant language
- Python
- Stars
- 65
- Forks
- 11
- Avg merge
- 21h 25m
- Merged PRs (30d)
- 70
Description
## Summary
The calculator's dropdown offers 334 stations from `/api/stations/`, but only 271 have usable pre-calculated percentile data. The remainder fall into two groups:
- 63 stations with no data file at all. These currently return a 500 (#5688).
- 10 stations whose data file exists but is entirely null, e.g. 321 GREENSTONE HUB, 3190 ROCKINGHAM. These return 200 and render every index as "Not available" with "Data only available for 0 of 10 years".
Even once the 500 is fixed, both groups are selectable options that cannot produce a result.
A station with no data file, selected alongside two that work (#5688):

A station whose data file is entirely null. No error, but no result either:

## The 63 with no data file
| Count | Category | Examples |
|------:|----------|----------|
| 20 | Environment Canada `(EC)` | TOFINO AP, VANCOUVER AP, VICTORIA AP |
| 11 | Other agency `(NC)` / `(MC)` / `(CRD)` | 31N (CRD) 543, 14G (CRD) 1142 |
| 12 | Test / project stations | QD 9 TEST, QD1 10M TEST, BLOWDOWN PASS PROJECT |
| 20 | Other | TERRACE, KOMIE, BLUEBERRY, COBBLE HILL |
Full list: 1, 2, 26, 44, 49, 69, 83, 88, 122, 142, 157, 186, 203, 205, 229, 395, 434, 435, 480, 543, 896, 915, 967, 1013, 1015, 1016, 1141, 1142, 1143, 1166, 1173, 1851, 3730, 3810, 3851, 3873, 4232, 4270, 4332, 4352, 4393, 4513, 4552, 4653, 4713, 4793, 4973, 5154, 5716, 5796, 5798, 5799, 5996, 5997, 6016, 6076, 6117, 6136, 6156, 6177, 6196, 6216, 6236
## Context that may be relevant
`app/data/weather_stations.json` is still in the repo and still contains exactly 286 entries, matching the 286 data files. The web app also still passes `source=unspecified` when calling `/api/stations/` (`PercentileCalculatorPage.tsx:43`), which the endpoint no longer reads; `StationSourceEnum` was removed in #4040. I have not assumed which behaviour is intended.
## Approaches
1. Restrict the calculator's dropdown to stations that have usable data. Needs some way for the web app to learn which codes are supported.
2. Regenerate the pre-calculated data to cover more stations. `RECENT_YEAR` in `calculate_percentile_offline.py` is currently 2023 and the script reads from an S3 bucket I do not have access to.
Whether these stations should have percentile data at all may be worth a conversation, given that a fair number are Environment Canada, other-agency, or test stations.
It is quite possible I am missing context here, and that some of these are offered deliberately. If so I would be glad to know why. Otherwise, happy to take a run at 1 if that is a direction you would like to go.
Contributor guide
Research direction
Start with PercentileCalculatorPage.tsx:43 and the /api/stations/ endpoint, then inspect app/data/weather_stations.json and the available percentile data files. Confirm with maintainers whether the dropdown should include only stations with usable data or whether missing stations are intentional. Done means the selected behavior is defined and unavailable stations no longer offer an unusable percentile result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fastapi, python, react, typescript
- Domain
- api, backend, frontend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100