CivicTechTO / CivicTechTO/toronto-bids
Validate the provisional bids&tenders portal parser once a portal has data
- Dominant language
- Python
- Stars
- 7
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
Follow-up from #135 / PR #139. The bids&tenders portal-listing capture is merged and armed, but `parse_listing` shipped **PROVISIONAL**: both TRCA and Zoo portals are currently empty (`total=0`, verified live), so it is mapped to the portal's grid-JS field names and validated only against a **synthetic** fixture (`tests/fixtures/agencies/bids_tenders_record_sample.json`, labelled synthetic in SOURCES.md).
**Trigger:** when a TRCA or Zoo bid first appears on its portal (check with `tb enrich-agencies --portal --record` — non-zero output means data exists).
**Then:**
1. `--record` captures the real JSON to `/agencies/portal_recordings/`; replace the synthetic fixture with a real one.
2. Fix `parse_listing` against real records and re-validate with recall **measured** against the endpoint's `total` (the #136 lesson — offline fixtures lied about real corpora three times; don't trust a synthetic one).
3. Settle the deferred questions the empty portals left open:
- the exact `status` code → Open/Awarded/Cancelled mapping;
- whether the portal bid number matches the board-report `native_ref` format (how often the COALESCE enrich fires vs. a distinct row);
- whether awarded records expose a supplier/value → build a portal `agency_award` path (currently not built);
- whether `store_listings` should overwrite `status` on a nightly re-fetch (it currently backfills with `overwrite=False`, so a stale status is not refreshed — the deliberate no-data-loss choice, revisit with real data).
Until then the nightly portal step is a verified clean no-op. Spec: `docs/superpowers/specs/2026-07-18-bidsandtenders-portal-listings-design.md` ("Deferred to first-real-data").
Contributor guide
No contributing guide indexed for this repository
Research direction
Wait until `tb enrich-agencies --portal --record` reports data for TRCA or Zoo, then capture a real record in `/agencies/portal_recordings/`. Read `docs/superpowers/specs/2026-07-18-bidsandtenders-portal-listings-design.md` and compare the real capture with `parse_listing` and `tests/fixtures/agencies/bids_tenders_record_sample.json`. Done means validation against the endpoint’s `total` and resolution of the deferred mapping, matching, award-data, and status-refresh questions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100