CivicTechTO / CivicTechTO/toronto-bids
Extraction client — text in, validated records out, refuse on malformed
- Dominant language
- Python
- Stars
- 7
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
> *This was generated by AI during triage.*
## Parent
#205
## What to build
A client module that takes document text and returns validated extraction records conforming to the schema in `docs/protocols/extraction-schema-by-document-type.md`. Two models: **Nemotron 3 Ultra (free)** as default, **GPT-5.6-Luna (~$4 for the full backlog)** as automatic fallback on rate-limit or unavailability. Via OpenRouter.
Four things that already bit during evaluation and must be handled:
1. **Retry with backoff.** Its absence scored Sol at 22% instead of 100% — four instant retries under rate limiting, silently returning empty.
2. **Validate-and-refuse for unconstrained models.** Nemotron has no structured-output support; it produces JSON by following instructions. Output must be schema-validated and a document refused rather than partially stored on malformed output.
3. **Provider quirks.** DeepSeek needs the literal word "json" in the prompt; Tencent ignores response schemas and returns markdown. Any production path needs that fallback logic or it will score a capable model as useless.
4. **Placeholder-credential guard.** `scrapers/.env.example` ships with `OPENROUTER_API_KEY=your-openrouter-api-key`; a checkout that copied it but never filled it in must fail immediately with a clear message, not 30 seconds later inside an HTTP call (#184 pattern).
`tb extract --dry-run ` prints validated records for a held document and writes nothing. Offline tests from recorded response fixtures (no network in tests — same discipline as the rest of the suite).
## Acceptance criteria
- [ ] `tb extract --dry-run ` produces validated records from a held document
- [ ] Nemotron default, Luna fallback on rate-limit or unavailability
- [ ] Malformed output is refused, not partially stored
- [ ] Placeholder API key detected and refused before any HTTP call
- [ ] Offline test suite using recorded response fixtures, no network
- [ ] `OPENROUTER_API_KEY` read from `scrapers/.env`, gitignored, never committed
## Blocked by
None — can start immediately (parallel with the extractor seam).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with docs/protocols/extraction-schema-by-document-type.md and inspect the existing extractor seam referenced by the parent issue. Review scrapers/.env.example and the #184 credential-guard pattern, then run the held-document tb extract --dry-run command and examine recorded response fixtures. Done means validated records are printed, malformed output and placeholder credentials are refused, fallback and retry behavior work, and tests make no network calls.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, cli, security, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100