aws / aws/bedrock-agentcore-sdk-python

[Enhancement] Evaluation Client: extensible hook for external reasoning verification before payment settlement

Ouverte
#466 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
761
Forks
147
Merge moyen
1 j 23 h
PR mergées (30 j)
7

Description

Following up on #393 (Evaluation Client — Lifecycle, Orchestration & Online Pipeline) — raising a specific design question before the interface solidifies.

**Context**

AgentCore Payments introduces a new failure mode that goes beyond what existing guardrails cover: a payment flow where the agent's reasoning was internally consistent but the *decision to transact* was poorly grounded. The AWS blog post acknowledged this directly under roadmap: *"stronger buyer intent verification."*

The current observability stack (logs, metrics, traces in the AgentCore console) captures *what happened* after the fact. What I'm not seeing is a hook for pre-settlement verification — a point in the execution loop where external logic can inspect the agent's reasoning trace and return a structured verdict before AgentCore finalizes the payment.

**Concrete ask**

When the Evaluation Client (#393) is designed, would it support:

1. A pre-settlement callback interface — e.g. `on_before_payment(trace, context) -> VerificationResult` — that can short-circuit the transaction if the reasoning doesn't meet a defined threshold?
2. A structured trace format that evaluation logic can consume deterministically (not just raw logs)?
3. A way to attach the verification result as metadata to the transaction record, so audit trails include both *what was paid* and *why the reasoning was considered sound*?

This pattern is especially relevant for regulated use cases (financial services, healthcare, high-stakes procurement) where "the agent decided to transact" is not sufficient — you need a provable record that the reasoning behind the decision was evaluated.

Happy to share a reference architecture sketch if it would help the design discussion — particularly around the trace-format and threshold-semantics questions.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez par examiner l’issue #393 et la discussion de conception de Evaluation Client, puis comparez le callback de pré-règlement demandé, la trace structurée et les métadonnées de transaction avec le flux AgentCore Payments décrit ici. Le travail sera considéré comme terminé lorsqu’une interface et une sémantique pour la vérification auront été définies, notamment la manière dont un verdict peut empêcher le règlement et apparaître dans l’enregistrement d’audit.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
aws, python
Domaine
ai, backend-api-design, payments
Type d'issue
Fonctionnalité
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
Calme
Clarté
Plutôt claire
Accessibilité débutants
38/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.