bcgov / bcgov/wps

Percentile Calculator offers stations it has no data for

Open
#5,689 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
65
Forks
11
Avg merge
21h 25m
Merged PRs (30d)
70

Description

## Summary

The calculator's dropdown offers 334 stations from `/api/stations/`, but only 271 have usable pre-calculated percentile data. The remainder fall into two groups:

- 63 stations with no data file at all. These currently return a 500 (#5688).
- 10 stations whose data file exists but is entirely null, e.g. 321 GREENSTONE HUB, 3190 ROCKINGHAM. These return 200 and render every index as "Not available" with "Data only available for 0 of 10 years".

Even once the 500 is fixed, both groups are selectable options that cannot produce a result.

A station with no data file, selected alongside two that work (#5688):

![500 error](https://raw.githubusercontent.com/Gabriel-Dalton/wps/assets/percentile-calculator-screenshots/.screenshots/03-error-poisons-batch.png)

A station whose data file is entirely null. No error, but no result either:

![no data rendering](https://raw.githubusercontent.com/Gabriel-Dalton/wps/assets/percentile-calculator-screenshots/.screenshots/04-blank-321.png)

## The 63 with no data file

| Count | Category | Examples |
|------:|----------|----------|
| 20 | Environment Canada `(EC)` | TOFINO AP, VANCOUVER AP, VICTORIA AP |
| 11 | Other agency `(NC)` / `(MC)` / `(CRD)` | 31N (CRD) 543, 14G (CRD) 1142 |
| 12 | Test / project stations | QD 9 TEST, QD1 10M TEST, BLOWDOWN PASS PROJECT |
| 20 | Other | TERRACE, KOMIE, BLUEBERRY, COBBLE HILL |

Full list: 1, 2, 26, 44, 49, 69, 83, 88, 122, 142, 157, 186, 203, 205, 229, 395, 434, 435, 480, 543, 896, 915, 967, 1013, 1015, 1016, 1141, 1142, 1143, 1166, 1173, 1851, 3730, 3810, 3851, 3873, 4232, 4270, 4332, 4352, 4393, 4513, 4552, 4653, 4713, 4793, 4973, 5154, 5716, 5796, 5798, 5799, 5996, 5997, 6016, 6076, 6117, 6136, 6156, 6177, 6196, 6216, 6236

## Context that may be relevant

`app/data/weather_stations.json` is still in the repo and still contains exactly 286 entries, matching the 286 data files. The web app also still passes `source=unspecified` when calling `/api/stations/` (`PercentileCalculatorPage.tsx:43`), which the endpoint no longer reads; `StationSourceEnum` was removed in #4040. I have not assumed which behaviour is intended.

## Approaches

1. Restrict the calculator's dropdown to stations that have usable data. Needs some way for the web app to learn which codes are supported.

2. Regenerate the pre-calculated data to cover more stations. `RECENT_YEAR` in `calculate_percentile_offline.py` is currently 2023 and the script reads from an S3 bucket I do not have access to.

Whether these stations should have percentile data at all may be worth a conversation, given that a fair number are Environment Canada, other-agency, or test stations.

It is quite possible I am missing context here, and that some of these are offered deliberately. If so I would be glad to know why. Otherwise, happy to take a run at 1 if that is a direction you would like to go.

Contributor guide

Open the contributing guide

Research direction

Start with PercentileCalculatorPage.tsx:43 and the /api/stations/ endpoint, then inspect app/data/weather_stations.json and the available percentile data files. Confirm with maintainers whether the dropdown should include only stations with usable data or whether missing stations are intentional. Done means the selected behavior is defined and unavailable stations no longer offer an unusable percentile result.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, python, react, typescript
Domain
api, backend, frontend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.