databrickslabs / databrickslabs/ontos
[Improvement]: Term Mapping bulk-accept should confirm/guard against low-confidence rows
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 212
- Forks
- 71
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 43
Description
Context
Surfaced while smoke-testing the Term Mapping workbench (PR #481, PRD #469). The bulk Accept N action in suggestion-queue-table.tsx accepts every selected row indiscriminately, regardless of confidence bucket. Combined with the DataTable's page-scoped "Select all" checkbox, the natural "select all → accept" gesture happily promotes 50% noise hits like `transaction_id → productId` or `store_id → orderId` into the accepted bucket where a single click on Apply can persist them as real semantic links.
The suggester is doing its job — it scores those low (50–70%) and surfaces a "Why" explanation — but the workbench gives no friction before turning that into a write.
Repro:
- Run `/term-mapping`, upload a small customer ontology, create a run with default filters.
- In the suggestion queue, click the header Select all checkbox (selects 10 rows on the first page).
- Click Accept 10.
- Observe: all 10 are accepted, including obvious mismatches with confidence in the 50–60% range.
Proposed UX
Tiered guard-rail, in order of effort:
- Minimum: when the selection contains rows below a configurable threshold (default 0.7), show a confirm dialog summarising the confidence distribution before submitting decisions, e.g. "Accept 10 selected: 1 high (≥80%), 2 medium (60–80%), 7 low (<60%). Continue?". Reuse the existing apply-dialog visual treatment.
- Better: default the suggestion table sort to confidence DESC and surface filter chips for High / Medium / Low so the obvious move becomes "accept all High".
- Best: add a one-click "Accept all ≥ X%" quick action above the table that operates on the full pending set, not just the current page (today's Select all is page-scoped, which is its own footgun).
Acceptance criteria
- Bulk-accepting a selection that contains any suggestion below the configured threshold prompts a confirmation summarising the per-bucket counts.
- User can dismiss the dialog (no decisions persisted) or proceed (existing POST behaviour).
- Threshold is sourced from the same scoring constants as the suggester (
AUTO_ACCEPT/ a newLOW_CONFIDENCEcutoff incontroller/term_mapping/scoring.py) so backend and UI stay in lockstep. - Page-scoped vs. full-pending "Select all" behaviour is either documented in a tooltip or replaced with a "Select all 18 pending" affordance.
Related
- PRD: #469
- Backend PR: #480
- UI PR: #481 (current implementation)
- Phase 4 (
LLMJudgeSuggesterre-ranker) will reduce the number of low-confidence noise candidates that ever reach the queue, but is complementary to this guard-rail rather than a replacement.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in suggestion-queue-table.tsx and trace the bulk Accept action and page-scoped selection behavior. Read controller/term_mapping/scoring.py for AUTO_ACCEPT or LOW_CONFIDENCE, then inspect the existing apply-dialog visual treatment. Done means low-confidence bulk selections prompt with per-bucket counts, dismissal persists nothing, proceeding keeps the existing POST behavior, and selection scope is documented or improved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, typescript
- Domain
- backend, frontend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100