dbt-labs / dbt-labs/dbt

info schema: resolve package-scoped vars in `dbt.project_vars`

Open
#16,042 0 comments 0 reactions 0 assignees View on GitHub
area:engine status:triage
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

Follow-up from dbt-labs/fs#13498, which reshaped `dbt.project_vars` to one row per `(project_name, var_name)`.

That fixed the headline bug in #15850 — `var_name` is now a real variable name and `project_name` a real project, rather than a row per package with the package name in `var_name`. One case is still wrong.

### What's still wrong

`vars_json` is `{package_name: }`. The outer key *is* the package, but the inner map is the **unresolved** `vars:` block from `dbt_project.yml`, so a package-scoped override stays nested inside it rather than becoming its own package.

Given:

```yaml
vars:
region: emea
retention_days: 90
dbt_utils:
region: apac
```

today's output is:

| project_name | var_name | var_value |
| -- | -- | -- |
| my_project | region | emea |
| my_project | retention_days | 90 |
| my_project | dbt_utils | `{"region":"apac"}` |

The last row is wrong — `dbt_utils` is a package scope, not a variable. #15850's expected output is one row per variable per scope, with globals inherited:

| project_name | var_name | var_value |
| -- | -- | -- |
| my_project | region | emea |
| my_project | retention_days | 90 |
| dbt_utils | region | apac |
| dbt_utils | retention_days | 90 |

### Why it wasn't fixed in the first PR

Telling a package-scope key apart from a variable whose value happens to be an object requires the set of installed packages. `dbt.packages` is available, but in the common case the scoped package isn't installed at all — at which point dbt itself would apply those vars to nothing, and treating the key as a plain object-valued variable is arguably the more correct reading.

So this needs two decisions rather than just an implementation:

1. What counts as a package scope — a key present in `dbt.packages`, or any nested map?
2. What happens to a scope naming a package that isn't installed — drop it, keep it as a variable, or emit it anyway?

Plus the inheritance rule: globals visible in every scope, overridden per scope.

### Where

`build_project_vars` in `fs/sa/crates/dbt-index-core/src/info_schema/mod.rs`. The adapter's built-in packages (`dbt`, `dbt_`, and the parent package where one exists) are already filtered out there.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.