PolicyEngine / PolicyEngine/policyengine-uk-chat

Exception-terminated chat turns are never billed

Open
#176 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1
Forks
1
Avg merge
16h 46m
Merged PRs (30d)
2

Description

Problem

#145 added usage recording to two terminal paths of the /chat/message stream that previously ended a turn without billing — client_disconnected (both checkpoints) and loop_detected. One unbilled terminal path remains: a turn that ends via an exception.

When generate_stream() terminates through the captured_exception path (or the outer except fallback), the turn has typically already consumed real Anthropic tokens — model calls and tool iterations happen before the failure — but no record_usage call is reached, so the turn is free.

Where

backend/chat/orchestrator.py, the two terminal error yields (currently ~lines 718 and 724 on main; #159 touches the same region to sanitize the streamed message, so land that first and locate the yields afterwards).

Notes

  • Flagged as non-blocking in the #145 review ("the exception/error termination path is the one terminal path still unbilled — not blocking; flagging so it's tracked").
  • Billing on the exception path needs the same exactly-once care that #145's review verified for disconnect/loop: no double-count when an exception follows partially-recorded iterations.
  • Related: #159 (sanitizes what the error paths stream; no billing changes).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in backend/chat/orchestrator.py at generate_stream() and the two terminal error yields around the captured_exception and outer except paths; review #145's usage-recording changes and land #159 first as noted. Done means exception-terminated turns record consumed usage exactly once, including failures after partially recorded iterations, without double-counting.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.