dbt-labs / dbt-labs/agents_schema

Unable to use environment scoped secrets

Open
#39 1 comment 1 reaction 1 assignee Claimed by @fivetran-lukealexander View on GitHub
Dominant language
Python
Stars
48
Forks
4
PR merge metrics
No merged PRs in 30d

Description

This is a great initiative - we have already integrated this project into our CI/CD pipeline and have immediately observed benefits. However it was necessary for us to use the CLI directly, as both the reuseable workflow and composite action are incompatible with our setup. Raising this issue as the README implies the reuseable workflows are the only entry point for the project.

## Reusable workflow incompatible with environment scoped GitHub variables/secrets
In our implementation we have environment scoped GitHub variables/secrets which we use with `job..environment`. e.g.
```
tst:
SNOWFLAKE_ACCOUNT
SNOWFLAKE_DATABASE
SNOWFLAKE_ROLE
SNOWFLAKE_USER
SNOWFLAKE_WAREHOUSE
SNOWFLAKE_PRIVATE_KEY_RAW
SNOWFLAKE_PRIVATE_KEY_PASSPHRASE
prd:
SNOWFLAKE_ACCOUNT
SNOWFLAKE_DATABASE
SNOWFLAKE_ROLE
SNOWFLAKE_USER
SNOWFLAKE_WAREHOUSE
SNOWFLAKE_PRIVATE_KEY_RAW
SNOWFLAKE_PRIVATE_KEY_PASSPHRASE
```
It is not possible to compose an environment scoped `WAREHOUSE_CREDENTIALS` secret in one job and input that secret to the reuseable workflow: `dbt-labs/agents_schema/.github/workflows/agents-schema-dbt.yml@v0.0.10`.

## Composite action incompatible with our `pyproject.toml`
We attempted to use the composite action `dbt-labs/agents_schema/.github/actions/agents-schema-dbt@v0.0.10` within a job of our workflow. In this job, we:
- scope GitHub environment
- setup a uv environment using our `pyproject.toml` which uses python 3.11
- install dbt Fusion
- download dbt dependencies
- run dbt parse using environment scoped variables/secrets
- compose `WAREHOUSE_CREDENTIALS` secret based on environment scoped variables/secrets
- use `agents-schema-dbt` action

However as the `action-schema-dbt` action is hardwired to python 3.12 is incompatible with our `pyproject.toml` which constrains the python version with `requires-python = ">=3.11,<3.12"`.

Consequently we are using the CLI directly. In our workflow we:
- scope GitHub environment
- setup a uv environment using our `pyproject.toml` which uses python 3.11
- install dbt Fusion
- download dbt dependencies
- run dbt parse using environment scoped variables/secrets
- compose `WAREHOUSE_CREDENTIALS` secret based on environment scoped variables/secrets
- `uvx --from "agents-schema==${AGENTS_SCHEMA_VERSION}" agents-schema dbt --project-dir "${DBT_PROJECT_DIR}"`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.