aws / aws/context-ontology-accelerator
[Bug]: The bundled live_induction_run.py does not work against the release it ships with
- Dominant language
- Python
- Stars
- 741
- Forks
- 82
- Avg merge
- 7h 21m
- Merged PRs (30d)
- 10
Description
# Description
Running the bundled `live_induction_run.py` against a deployed namespace, the way
`packages/ontology-engine/README.md` directs, stops at the accept step with
`TypeError: 'AcceptProposalResponse' object is not subscriptable`. Working around that,
the run completes and reports grounding results — for grounding it never attempted.
Every table logs an empty grounding scope, and the run still reads as a success.
It is the documented starting point for a non-interactive induction run, and it does
not complete against the release it ships with.
The bundled scripts should run against the release they ship with, and a run that
never attempted grounding should not read like one that did.
## Environment
- **Commit ID:** `b24e4b2`; the script is unchanged since `v0.2.1`
- **Environment:** first reported on `v0.2.1` in `us-east-1`
## Step to reproduce
- [ ] `python scripts/live_induction_run.py` against a deployed namespace, the way
`packages/ontology-engine/README.md` directs
- [ ] It stops at the accept step with
`TypeError: 'AcceptProposalResponse' object is not subscriptable`
- [ ] Work around it by polling for the terminal state, then read the
`grounding_recall` log line for each table
## A root cause
The accept call returns an `AcceptProposalResponse` object with `status:
"accepting"`, and the script subscripts the response and asserts `status ==
"accepted"`. On a second accept the API returns `"accepted"` where the script expects
`"already_accepted"`.
`grounding_ontology_ids` is left unset. Every table then logs
`grounding_recall: empty grounding scope — returning no candidates (all-novel)`, and
the run completes and reports results.
## Context
Reported by an external evaluation team.
Accept:
TypeError: 'AcceptProposalResponse' object is not subscriptable
live_induction_run.py:610 -> assert a["status"] == "accepted"
The response `status` is `"accepting"` and the script does not poll for a terminal
state. On a second accept it expects `"already_accepted"` while the API returns
`"accepted"`. They worked around it by polling DynamoDB.
Grounding: every table logs
grounding_recall: empty grounding scope — returning no candidates (all-novel)
The run completes and reports results, so it reads as though grounding had been
verified. With the scope passed explicitly, recall returned about 30 candidates per
table and the reranker selected `NONE` with a stated reason — the mechanism works and
the harness was not engaging it.
The scripts in that directory are not exercised by CI.
Contributor guide
Research direction
Start at scripts/live_induction_run.py around line 610 and compare its accept handling with the AcceptProposalResponse behavior described in the issue; read the invocation documented in packages/ontology-engine/README.md. Exercise the script against a deployed namespace, including a second accept and grounding logs. Done means it completes against its shipped release and distinguishes an unattempted grounding scope from verified results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- backend, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100