[Bug] If a test / model config is added (to a `schema.yml`) before a model is written (`model.sql`), partial parsing will not find the test / model config until a full reparse
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this a new bug in dbt-core?
- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
If you wrote a test first, parse the project, then wrote the model, dbt will not actually run the test due to partial parsing.
### Expected Behavior
Even with partial parsing, dbt should know to run the test even if written prior to the model file.
### Steps To Reproduce
1. Add a schema yml file first but importantly don't create any model file yet.
```yaml
# models/schema.yml
models:
- name: foo
columns:
- name: id
data_tests:
- not_null
```
2. Do initial parse with clean `target/`:
```sh
$ ls target
ls: target: No such file or directory
$ dbt parse
20:30:13 Running with dbt=1.8.3
20:30:13 Registered adapter: postgres=1.8.2
20:30:13 Unable to do partial parsing because saved manifest not found. Starting full parse.
20:30:14 [WARNING]: Did not find matching node for patch with name 'foo' in the 'models' section of file 'models/schema.yml'
20:30:14 [WARNING]: Test 'test.my_dbt_project.not_null_foo_id.f099b1e59c' (models/schema.yml) depends on a node named 'foo' in package '' which was not found
20:30:14 [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
There are 1 unused configuration paths:
- models.my_dbt_project
20:30:14 Performance info: /Users/jeremy/git/dbt-basic/target/perf_info.json
```
3. Add model file:
```sql
-- models/foo.sql
select 1 id
```
4. Build:
```sh
$ dbt build
20:31:35 Running with dbt=1.8.3
20:31:35 Registered adapter: postgres=1.8.2
20:31:35 Found 1 model, 413 macros
20:31:35
20:31:35 Concurrency: 4 threads (target='pg')
20:31:35
20:31:35 1 of 1 START sql table model public.foo ........................................ [RUN]
20:31:35 1 of 1 OK created sql table model public.foo ................................... [SELECT 1 in 0.07s]
20:31:35
20:31:35 Finished running 1 table model in 0 hours 0 minutes and 0.19 seconds (0.19s).
20:31:35
20:31:35 Completed successfully
20:31:35
20:31:35 Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
```
^ Notice no "test" was identified.
5. Build but do a full parse:
```sh
$ dbt build --no-partial-parse
20:32:22 Running with dbt=1.8.3
20:32:22 Registered adapter: postgres=1.8.2
20:32:23 Found 1 model, 1 test, 413 macros
20:32:23
20:32:23 Concurrency: 4 threads (target='pg')
20:32:23
20:32:23 1 of 2 START sql table model public.foo ........................................ [RUN]
20:32:23 1 of 2 OK created sql table model public.foo ................................... [SELECT 1 in 0.06s]
20:32:23 2 of 2 START test not_null_foo_id .............................................. [RUN]
20:32:23 2 of 2 PASS not_null_foo_id .................................................... [PASS in 0.04s]
20:32:23
20:32:23 Finished running 1 table model, 1 test in 0 hours 0 minutes and 0.24 seconds (0.24s).
20:32:23
20:32:23 Completed successfully
20:32:23
20:32:23 Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2
```
### Relevant log output
_No response_
### Environment
```markdown
- OS: macOS
- Python: 3.11.9
- dbt: 1.8.3
```
### Which database adapter are you using with dbt?
postgres
### Additional Context
In dbt Cloud, every save action does a parse. So if a user was practicing TDD - I guess write the test first :P then they could run into this scenario.
https://dbt-labs.slack.com/archives/C02SRNY2EQ4/p1721245567642059
Contributor guide
Assessment
This issue has not been assessed yet.