dbt-labs / dbt-labs/dbt-adapters

[Bug] All test failures are stored even when a limit config is set

Open
#373 6 comments 1 reaction 0 assignees View on GitHub
type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-core?

- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

_Hi there, firstly thanks for the amazing job you're doing, I am huge fan of dbt !_

I use dbt core with dbt-redshift.

I wrote a test on a simple seed to check the behavior of the _store_failures_ test config when used in addition to the _limit_ config.

The seed I test (`seeds/customers.csv`) looks like this and has 100 rows (from ID 1 to 100):
```csv
ID,FIRST_NAME,LAST_NAME
1,Michael,P.
2,Shawn,M.
3,Kathleen,P.
4,Jimmy,C.
5,Katherine,R.
6,Sarah,R.
7,Martin,M.
8,Frank,R.
9,Jennifer,F.
10,Henry,W.
```
Download file here : [customers.csv](https://github.com/dbt-labs/dbt-core/files/14063939/customers.csv)

The test definition (`seeds/_seeds.yml`) looks like this:
_TLDR : The "ID" column must be strictly lower than 50. Store the failures. But stop if more than 20 failures._

```yaml
version: 2

seeds:
- name: customers
+quote_columns: false
columns:
- name: ID
tests:
- dbt_utils.accepted_range:
name: my_test_with_limit_and_store_failures
max_value: 50
inclusive: true
config:
store_failures: true
warn_if: "<5"
error_if: ">=5"
limit: 20
```

What I experience when running `dbt test -s customers` :
- The test log seems OK
![image](https://github.com/dbt-labs/dbt-adapters/assets/55416733/e6e45a55-9a62-4233-85b7-d1d7e7b20c0f)

- But my test audit table generated in my warehouse contains all the failed row (50) and not 20 (the limit specified).
![image](https://github.com/dbt-labs/dbt-adapters/assets/55416733/541b1a60-e2bd-4e08-941d-d458fcc93dc7)

### Expected Behavior

I guess only the 20 first row must be stored in the audit table (based on the [store_failure config documentation](https://docs.getdbt.com/reference/resource-configs/store_failures) pictured here)

![image](https://github.com/dbt-labs/dbt-adapters/assets/55416733/8985e18d-a54c-4352-b01e-ab8c23417caa)

### Steps To Reproduce

```bash
python3 -m venv .venv.
.venv/bin/activate; pip install -r requirements.txt
dbt deps
dbt seed
dbt test -s customers
```

Where requirements.txt is

```txt
dbt-core==1.7.6
dbt-redshift==1.7.2
```

### Relevant log output

_No response_

### Environment

```markdown
- OS:Ubuntu 22.04.3 LTS
- Python: 3.9.7
- dbt: 1.7.6
```

### Which database adapter are you using with dbt?

redshift

### Additional Context

I noticed this SQL file generated in the target folder if it can help

![image](https://github.com/dbt-labs/dbt-adapters/assets/55416733/97be0818-11c9-4594-9476-35a5ca16ed4f)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.