dbt-labs / dbt-labs/dbt

[CT-997] [Feature] Support dbt.var + dbt.env_var in Python models

Open
#5,617 4 comments 1 reaction 0 assignees View on GitHub
engine:v1 python_models type:feature
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this your first time submitting a feature request?

- [X] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

Copying from [thread in #beta-feedback-python-models](https://getdbt.slack.com/archives/C03QUA7DWCW/p1659470042461869):

> A quick thought, why not expose a `dbt.var` function (and similar for env var) rather than overloading config? If you apply the same limitations as for config function (ie. Only literal args) should be pretty straightforward to statically analyse which vars need to be injected?

It's true! The mechanism for `dbt.var` and `dbt.env_var` could be just the same as `dbt.config.get`. We detect the literal arguments passed in:
https://github.com/dbt-labs/dbt-core/blob/eb72dbf32aa96394d599c28db439e0b2c6fcc135/core/dbt/parser/models.py#L113-L117

https://github.com/dbt-labs/dbt-core/blob/eb72dbf32aa96394d599c28db439e0b2c6fcc135/core/dbt/parser/models.py#L182-L184

And then provide those values at "compile" time (which we're currently doing via Jinja... gross), as new methods (`var` + `env_var`) on the `dbtObj` class:

https://github.com/dbt-labs/dbt-core/blob/eb72dbf32aa96394d599c28db439e0b2c6fcc135/core/dbt/include/global_project/macros/python_model/python.sql#L33-L40

https://github.com/dbt-labs/dbt-core/blob/eb72dbf32aa96394d599c28db439e0b2c6fcc135/core/dbt/include/global_project/macros/python_model/python.sql#L71-L77

### Describe alternatives you've considered

Not doing this, and continuing to ask users (as we are currently) to pass those vars + env_vars in as configs in `.yml` files:
```yml
version: 2

models:
- name: my_python_model
config:
materialized: table
target_name: "{{ target.name }}"
specific_var: "{{ var('SPECIFIC_VAR') }}"
specific_env_var: "{{ env_var('SPECIFIC_ENV_VAR') }}"
```

### Who will this benefit?

Usability and ergonomics for people developing and configuring Python models, especially ones with more advanced conditional logic or env-specific behavior

### Are you interested in contributing this feature?

_No response_

### Anything else?

To be clear, the environment variable values would need to be the ones evaluated during _dbt parsing_. (They are NOT the environment variables set in the runtime environment, e.g. Snowpark.)

That's slightly different from how environment variables work in SQL models today, since the `{{ env_var() }}` Jinja is re-rendered at compile/execute runtime.

But I think it probably _should_ work this way across the board, where all inputs (including env vars) are resolved during parsing and pre-execution.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.