OpenFn / OpenFn/apollo

Sentry: scrub the exception chain, and give operators a channel that still has detail

Open
#656 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
5
Forks
10
Avg merge
2d 20h
Merged PRs (30d)
17

Description

Two things left over from the #446 review that are real but do not belong in that branch.

Job bodies still reach Sentry through the exception chain

services/entry.py sets include_local_variables=False, and the comment above it names __cause__/__context__ walking as the reason. But disabling frame locals does not stop Sentry serialising the chained exception itself: entry.py:133 raises inside an except block, so the original exception travels as __context__ and its message goes to Sentry even though no frame does. For a PyYAML error that message quotes the offending line of workflow_yaml. The comment therefore claims a property the code does not have, which is worse than not having it.

The fix is a before_send that walks the chain and scrubs, rather than relying on any single flag. This is the same scrubber #654 needs for set_context, so the two should be done together.

Failure detail is now lost on every channel

The #446 branch removed exception text from every log line and error payload on the chat path, which was right as security and leaves operators with nothing. An operator seeing The workflow agent failed (RuntimeError) has the type and no stack, no message and no context, and there is no capture_exception anywhere in the global_chat path to make up for it. We traded one problem for another.

The honest fix is a scrubbed operator channel: capture_exception on the chat path once before_send is trusted to scrub, so Sentry gets the detail and the SSE stream does not. Until then, a production failure in the planner or a subagent is close to undebuggable from the logs alone.

Related: #654 (the set_context, llm_edit_answer and LoggingIntegration routes) and #655 (the Unicode gating work). The scrubber is the common dependency for this issue and #654.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in services/entry.py, especially the Sentry setup and the raise at line 133, then trace the global_chat path and the related #654 routes. Done means chained exception data is scrubbed before Sentry receives it, while the chat path captures scrubbed failure detail without exposing it through the SSE stream.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, observability, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.