traceloop / traceloop/openllmetry

๐Ÿ› Bug Report: gen_ai.bedrock.guardrail.activation counts every Bedrock call

Open Beginner friendly
#4,471 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.4k
Forks
1.1k
Avg merge
8d 14h
Merged PRs (30d)
2

Description

Component: Bedrock Instrumentation

๐Ÿ“œ Description

is_guardrail_activated() in guardrail.py ends with:

return response.get("amazon-bedrock-guardrailAction") != "NONE"

When no guardrail is configured, Bedrock omits that key, so .get() returns None and None != "NONE" is True. Every ordinary response is treated as a guardrail activation, and gen_ai.bedrock.guardrail.activation is incremented on every Bedrock call.

Defaulting the key fixes it: response.get("amazon-bedrock-guardrailAction", "NONE") != "NONE".

๐Ÿ‘Ÿ Reproduction steps
from opentelemetry.instrumentation.bedrock.guardrail import is_guardrail_activated

print(is_guardrail_activated({"stopReason": "end_turn"}))  # True, expected False
๐Ÿ‘ Expected behavior

A response with no amazon-bedrock-guardrailAction key is not an activation.

๐Ÿ‘Ž Actual Behavior

It counts as one, so the metric reports activations for users with no guardrails configured.

๐Ÿค– Python Version

3.12

๐Ÿ“ƒ Additional context

The same line crashes when metrics are disabled, since _instrument() sets the counters to None but guardrail.py dereferences them unconditionally:

guardrail.py", line 179, in guardrail_converse
    metric_params.guardrail_activation.add(1, attrs)
AttributeError: 'NoneType' object has no attribute 'add'

Present from 0.39.0 through 0.62.3.

๐Ÿ‘€ Duplicate check

Checked, no existing issue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up โ€” it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in guardrail.py with is_guardrail_activated() and the guardrail_converse path described in the issue. Run the provided reproduction and inspect how metrics-disabled instrumentation reaches guardrail_activation.add. Done means responses without the guardrail action key are not activations and disabled metrics do not dereference a missing counter.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
observability
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.