dbt-labs / dbt-labs/dbt

[Bug] Lack of migration path from user configs defined in `profiles.yml` to the 'flags' key in `dbt_project.yml`

Open
#10,402 4 comments 4 reactions 0 assignees View on GitHub
engine:v1 type:bug
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this a new bug in dbt-core?

- [X] I believe this is a new bug in dbt-core
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

Initially introduced with https://github.com/dbt-labs/dbt-core/pull/9289 and released with dbt-core v1.8 (though I see from https://github.com/dbt-labs/dbt-core/issues/9183 that it was also backported to 1.6 and 1.7), there isn't a viable migration path for moving from user configs defined in `profiles.yml` to the 'flags' key in dbt_project.yml.

The current behavior is that if user configs ARE defined in `profiles.yml` and the 'flags' key is NOT defined dbt_project.yml, dbt will yield this **warning**:
```
[WARNING]: Deprecated functionality

User config should be moved from the 'config' key in profiles.yml to the 'flags' key in dbt_project.yml.
```

if user configs ARE defined in `profiles.yml` and the 'flags' key is ALSO defined in `dbt_project.yml`, dbt will yield this **error**:
```
Runtime Error
Do not specify both 'config' in profiles.yml and 'flags' in dbt_project.yml. Using 'config' in profiles.yml is deprecated.
```

The issue is that, [as mentioned in dbt's documentation](https://docs.getdbt.com/docs/core/connect-data-platform/connection-profiles), `profiles.yml` is often not part of the dbt project and so not version-controlled:
>This file generally lives outside of your dbt project to avoid sensitive credentials being checked in to version control

For an organization to make this change, then, each member of the organization would need to update their `profiles.yml` simultaneously with the creation of the `flags` key in `dbt_project.yml` in order to avoid a runtime error.

### Expected Behavior

My expectation is that setting both user configs defined in `profiles.yml` and the 'flags' key in dbt_project.yml would yield a deprecation _warning_, with the user config defined in `profiles.yml` being ignored in favor of the 'flags' key in `dbt_project.yml`. This would allow users to opt-in to the new behavior, as intended, instead of making this, effectively, a hard deprecation for any organization with more than one user.

### Steps To Reproduce

1. Use dbt-core v1.8.x
2. Define both a user config in `profiles.yml`:

```
config:
use_colors: false
```
and a 'flags' key in `dbt_project.yml`
```
flags:
use_colors: false
```
3. Run any dbt command (say, `dbt debug`)
4. See runtime error
```
Runtime Error
Do not specify both 'config' in profiles.yml and 'flags' in dbt_project.yml. Using 'config' in profiles.yml is deprecated.
```

### Relevant log output

_No response_

### Environment

```markdown
- OS: macOS-14.5-x86_64
- Python: 3.11.4
- dbt: 1.8.3
```

### Which database adapter are you using with dbt?

snowflake

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.