PolicyEngine / PolicyEngine/policyengine-uk-chat
Exception-terminated chat turns are never billed
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 16h 46m
- Merged PRs (30d)
- 2
Description
Problem
#145 added usage recording to two terminal paths of the /chat/message stream that previously ended a turn without billing — client_disconnected (both checkpoints) and loop_detected. One unbilled terminal path remains: a turn that ends via an exception.
When generate_stream() terminates through the captured_exception path (or the outer except fallback), the turn has typically already consumed real Anthropic tokens — model calls and tool iterations happen before the failure — but no record_usage call is reached, so the turn is free.
Where
backend/chat/orchestrator.py, the two terminal error yields (currently ~lines 718 and 724 on main; #159 touches the same region to sanitize the streamed message, so land that first and locate the yields afterwards).
Notes
- Flagged as non-blocking in the #145 review ("the exception/error termination path is the one terminal path still unbilled — not blocking; flagging so it's tracked").
- Billing on the exception path needs the same exactly-once care that #145's review verified for disconnect/loop: no double-count when an exception follows partially-recorded iterations.
- Related: #159 (sanitizes what the error paths stream; no billing changes).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in backend/chat/orchestrator.py at generate_stream() and the two terminal error yields around the captured_exception and outer except paths; review #145's usage-recording changes and land #159 first as noted. Done means exception-terminated turns record consumed usage exactly once, including failures after partially recorded iterations, without double-counting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100