dbt-labs / dbt-labs/dbt-project-evaluator
Rules aggregate by resource_name across packages, causing false positives
- Dominant language
- Shell
- Stars
- 577
- Forks
- 93
- Avg merge
- 9h 46m
- Merged PRs (30d)
- 1
Description
## Describe the bug
When a dbt project contains multiple packages that define resources with the same `resource_name`, some `dbt_project_evaluator` rules appear to aggregate across packages and produce false positives.
From reading the package code, it looks like several DAG-based rules operate on `resource_name` rather than a package-scoped identifier. As a result, resources from different packages that happen to share the same name can be treated as the same node.
## Examples of affected behavior
I confirmed this with examples shaped like the following:
- Package A has a staging model `stg_shared_name` that depends on an internal base model.
- Package B has another staging model with the same name `stg_shared_name`, but this one depends directly on a source.
In this setup, `Direct Join to Source` appears to report `stg_shared_name` even though the Package A model does not directly depend on a source.
I also confirmed a similar pattern for `Source Fanout`:
- Package A has a source `shared_source.table_x` and a model `stg_metrics`
- Package B has a source with the same name `shared_source.table_x` and a model with the same name `stg_metrics`
Each package individually has a 1:1 source-to-model relationship, but `Source Fanout` appears to aggregate them together and report a false positive.
I suspect `Models with Too Many Joins` and possibly other rules that aggregate by `resource_name` may be affected as well, although I have only directly verified the two cases above.
## Expected behavior
Resources defined in different packages should be treated as distinct graph nodes, even if they share the same `resource_name`.
## Actual behavior
Some rules appear to aggregate across packages based on `resource_name`, which causes false positives when identical names exist in multiple packages.
## Steps to reproduce
A minimal reproduction would look like this:
1. Create two dbt packages in the same project.
2. In both packages, define resources with the same model name and/or source name.
3. Make their lineage different across packages.
4. Run `dbt_project_evaluator`.
## Why this seems to happen
From reading the implementation, it looks like graph/resource models derive and propagate `resource_name`, and some downstream rules group or join on that value. If package identity is not included in the effective key, same-named resources across packages can collide.
## Impact
This makes it difficult to rely on evaluator output in monorepos or multi-package dbt projects where naming conventions are reused across packages.
## Workaround
As a local workaround, we filtered specific rules after `evaluation_results`, but that only helps for some false positives and does not solve cases where aggregated metrics (for example join counts) are already incorrect upstream.
## Environment
- dbt version: 1.11.7
- adapter: BigQuery
- dbt_project_evaluator version: current release in use
Contributor guide
Assessment
This issue has not been assessed yet.