kestra-io / kestra-io/plugin-dbt

Add dbt seeds and tests to the asset graph built by assets.enableAuto (currently models only)

Open
#319 1 comment 0 reactions 1 assignee Claimed by @Malaydewangan09 View on GitHub
area/plugin kind/customer-request
Dominant language
Java
Stars
14
Forks
14
Avg merge
1d 10h
Merged PRs (30d)
10

Description

### Feature description

`assets.enableAuto: true` on a dbt Cloud task creates one asset per dbt model and skips every other node type. Seeds and tests never reach the asset graph.

## Evidence

`CheckStatus` with `assets.enableAuto: true`, pointed at a completed dbt Cloud run.

dbt's own parse line for the project:

```
Found 7 models, 12 data tests, 5 seeds, 4 sources, 540 macros
```

Assets created: 7, all models.

```
angelic-lattice-394708.demo_data_marts.dim_customers Table
angelic-lattice-394708.demo_data_marts.dim_products Table
angelic-lattice-394708.demo_data_marts.fct_orders Table
angelic-lattice-394708.demo_data_staging.stg_customers External
angelic-lattice-394708.demo_data_staging.stg_orders External
angelic-lattice-394708.demo_data_staging.stg_payments External
angelic-lattice-394708.demo_data_staging.stg_products External
```

17 of the 24 model, seed and test nodes are absent from the catalog: all 12 tests and all 5 seeds.

## Why it matters

- Seeds are the raw layer. Without them the graph has no entry point: staging models appear to depend on nothing, so lineage stops one hop short of the source.
- Tests are the natural asset-check analogue. Without them an asset can read as fresh while its tests are failing.
- Prior art: Dagster's dbt integration groups seeds as a `raw` layer and surfaces dbt tests as asset checks on their model. Users coming from Dagster expect that shape.

## Proposed

`enableAuto` emits an asset per seed alongside models, and represents tests against the model they belong to: as a check if supported, or at minimum as metadata on that model's asset. All three node types are already present in the `manifest.json` and `run_results.json` the task parses.

## Related

#38 asks for dbt models and tests in the Execution Topology view. This is the asset-graph counterpart of the same gap.

## Environment

Kestra 2.0.0-SNAPSHOT EE, plugin-dbt 1.8.0, dbt Cloud multi-tenant (us1 cell).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.