aws / aws/context-ontology-accelerator

[Bug]: The bundled live_induction_run.py does not work against the release it ships with

Open
#175 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
741
Forks
82
Avg merge
7h 21m
Merged PRs (30d)
10

Description

# Description

Running the bundled `live_induction_run.py` against a deployed namespace, the way
`packages/ontology-engine/README.md` directs, stops at the accept step with
`TypeError: 'AcceptProposalResponse' object is not subscriptable`. Working around that,
the run completes and reports grounding results — for grounding it never attempted.
Every table logs an empty grounding scope, and the run still reads as a success.

It is the documented starting point for a non-interactive induction run, and it does
not complete against the release it ships with.

The bundled scripts should run against the release they ship with, and a run that
never attempted grounding should not read like one that did.

## Environment

- **Commit ID:** `b24e4b2`; the script is unchanged since `v0.2.1`
- **Environment:** first reported on `v0.2.1` in `us-east-1`

## Step to reproduce

- [ ] `python scripts/live_induction_run.py` against a deployed namespace, the way
`packages/ontology-engine/README.md` directs
- [ ] It stops at the accept step with
`TypeError: 'AcceptProposalResponse' object is not subscriptable`
- [ ] Work around it by polling for the terminal state, then read the
`grounding_recall` log line for each table

## A root cause

The accept call returns an `AcceptProposalResponse` object with `status:
"accepting"`, and the script subscripts the response and asserts `status ==
"accepted"`. On a second accept the API returns `"accepted"` where the script expects
`"already_accepted"`.

`grounding_ontology_ids` is left unset. Every table then logs
`grounding_recall: empty grounding scope — returning no candidates (all-novel)`, and
the run completes and reports results.

## Context

Reported by an external evaluation team.

Accept:

TypeError: 'AcceptProposalResponse' object is not subscriptable
live_induction_run.py:610 -> assert a["status"] == "accepted"

The response `status` is `"accepting"` and the script does not poll for a terminal
state. On a second accept it expects `"already_accepted"` while the API returns
`"accepted"`. They worked around it by polling DynamoDB.

Grounding: every table logs

grounding_recall: empty grounding scope — returning no candidates (all-novel)

The run completes and reports results, so it reads as though grounding had been
verified. With the scope passed explicitly, recall returned about 30 candidates per
table and the reranker selected `NONE` with a stated reason — the mechanism works and
the harness was not engaging it.

The scripts in that directory are not exercised by CI.

Contributor guide

Open the contributing guide

Research direction

Start at scripts/live_induction_run.py around line 610 and compare its accept handling with the AcceptProposalResponse behavior described in the issue; read the invocation documented in packages/ontology-engine/README.md. Exercise the script against a deployed namespace, including a second accept and grounding logs. Done means it completes against its shipped release and distinguishes an unattempted grounding scope from verified results.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
backend, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.