dbt-labs / dbt-labs/dbt

[Feature] Test run results should include failure value, even if not fail or warning

Open
#11,312 2 comments 1 reaction 0 assignees View on GitHub
dbt tests engine:v1 type:bug
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this your first time submitting a feature request?

- [x] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [x] I have searched the existing issues, and I could not find an existing issue for this feature
- [x] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

Currently the RunResult of data test will report failures at 0 unless there is a warning or error. [[code](https://github.com/dbt-labs/dbt-core/blob/aa89740311061c2e5a3a8dafc205209067c19b32/core/dbt/task/test.py#L299C9-L299C21)]

The scenario: I have `severity: warn` and `warn_if: '> 10'`, my data test returns 4 failures, my RunResult shows 0 failures.
I would expect the 4 to go through.

What would be the problem if we always returned the "failures" column of the query?

### Describe alternatives you've considered

Alternative to getting the actual failure result is to use the adapter response, get a reference to the job id (BigQuery) and look up the job result manually. A lot of effort for getting number correctly.

### Who will this benefit?

Anyone who uses RunResults, either via the produced json file or programmatically, to monitor their data workflows to some degree.

### Are you interested in contributing this feature?

Yes

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.