crewAIInc / crewAIInc/crewAI

[BUG] GuardrailResult accepts payloads for both outcomes

Open Beginner friendly
#7,565 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Description

GuardrailResult does not reliably enforce the outcome invariant documented by its fields: a successful result may also carry an error, and a failed result may also carry a result payload.

The validator runs separately for result and error. Because Pydantic validates fields in declaration order, error is not available while result is validated, so success=True, result=..., error=... is accepted. The validator also uses truthiness, allowing contradictory falsey payloads such as error="" or result=0.

Steps to Reproduce
from crewai.utilities.guardrail import GuardrailResult

print(GuardrailResult(success=True, result="accepted", error="rejected"))
print(GuardrailResult(success=True, result="accepted", error=""))
print(GuardrailResult(success=False, result=0, error="rejected"))

All three currently construct successfully.

Expected behavior

A successful GuardrailResult must not populate error, and a failed result must not populate result. Presence should be checked with is not None, so falsey values on the selected side remain valid.

The existing allowed states should remain valid, including:

GuardrailResult(success=True, result="")
GuardrailResult(success=True, result=None)
GuardrailResult(success=False, error="")
GuardrailResult(success=False, error=None)
Screenshots/Code snippets

The current field validator depends on info.data and truthiness:

@field_validator("result", "error")
def validate_result_error_exclusivity(cls, v, info):
    ...
Operating System

macOS (15.5)

Python Version

3.13

crewAI Version

Current main at 5c33fe4c713a.

crewAI Tools Version

1.15.22

Virtual Environment

Venv (uv)

Evidence

The validator has existed since GuardrailResult was introduced. Repository production paths construct results through from_tuple(), which already populates only the selected side, but GuardrailResult is a public model and its advertised validation invariant does not hold for direct construction or model validation.

Targeted searches for GuardrailResult, result/error exclusivity, and the validation message found no matching open issue or pull request. Existing LLM guardrail issues concern provider exceptions and retry behavior, not this model invariant.

Possible Solution

Replace the order-dependent field validator with an after model validator that rejects only the field opposite to success when it is not None. Add regressions for ordinary and falsey contradictions plus from_tuple() compatibility.

Additional context

AI-assisted report and proposed fix. Please apply the required llm-generated label; external contributors cannot add repository labels directly.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the GuardrailResult model and its from_tuple() construction path, then inspect the existing result/error validator. Add regressions for ordinary and falsey contradictory payloads while preserving the listed valid states and from_tuple() compatibility, then run the relevant GuardrailResult tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.