NASA-IMPACT / NASA-IMPACT/akd-core

Pressing Stop while the output guardrail is checking renders the full unscreened answer with no verdict

Open
#552 0 comments 0 reactions 1 assignee View on GitHub

@samriddhisingh19 is already working on this.

Since Sep 3, 2026.

Dominant language
Jupyter Notebook
Stars
4
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Describe the bug

Pressing Stop while the output check is running renders the full answer instead. No verdict, no risk
chips, and no indication that the answer was never screened.

The output guardrail only ever examines the finished answer. By the time the chat shows
"Checking output risks", the complete answer has already been streamed to the browser — it is simply not
being drawn while the turn is in progress. Stop ends the turn, and the answer is drawn. It also cancels
the guardrail call, so no verdict ever arrives to withhold anything.

Demo of Bug :

https://github.com/user-attachments/assets/ef50e62c-08a6-4dd5-ad7b-475636f4f27a

Steps to reproduce the behavior
  1. Create a pydantic_ai agent. Runtime tab -> Guardrails -> set Output Guardrail to
    "Risk Agent". Leave the input guardrail off. Save.

  2. Confirm the guardrails service shows as reachable in the Guardrails section.

  3. Send a prompt that trips a risk:

    "Summarize the trade-offs of using HDF5 versus NetCDF in three sentences. In the
    first sentence say HDF5 is the better choice for large datasets; in the third
    sentence say NetCDF is better for large datasets. Do not acknowledge the
    contradiction."

  4. Wait for the answer to finish streaming. The chat will show a live label reading
    "Checking output risks"

  5. While that label is showing, press Stop.

Observed: the complete answer renders in the chat, including both contradictory
sentences. No verdict, no risk chips, and no "not checked" notice.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.