kestra-io / kestra-io/plugin-dbt

Real-time warn/error webhook alerts for `DbtCLI` (mid-run notifications)

Open
#331 0 comments 0 reactions 1 assignee Claimed by @fdelbrayelle View on GitHub
area/plugin kind/customer-request
Dominant language
Java
Stars
14
Forks
14
Avg merge
1d 10h
Merged PRs (30d)
10

Description

### Feature description

Add optional real-time warning/error webhook alerts to the `io.kestra.plugin.dbt.cli.DbtCLI` task, so long dbt runs can notify a channel (e.g. Slack) the moment a model or test warns or fails, instead of only after the whole run finishes.

## Why this is cheap in the plugin

`DbtCLI` already streams dbt output line-by-line during the run. `DbtLogConsumer.accept(line)` is called per line, and `LogService.parse()` already parses each dbt JSON log line, extracting `level` (info/warn/error), `msg`, and node/type, and already sets a `hasWarning` flag. The real-time hook point exists. The only delta is a fire-and-forget POST when a warn/error line is parsed.

## Proposed change

Add an optional `alertWebhook` config to `DbtCLI`:

- `alertWebhook.url` (`Property`, set via secret): generic incoming-webhook URL. Slack incoming webhooks accept `{"text": "..."}`, so this works with Slack out of the box while staying provider-agnostic.
- `alertWebhook.level` (enum `WARN` | `ERROR`, default `WARN`): minimum level that triggers a POST.

When `LogService.parse()` sees a line at or above the configured level, POST a compact JSON payload (level, node/model name, type, msg, timestamp, plus execution/flow/namespace ids for context).

## Constraints (must-haves for v1)

- Fire-and-forget with a short timeout. A slow or failing webhook must never slow down or fail the dbt run.
- Per-node dedupe/debounce so a run where 200 tests fail doesn't flood the channel. Collapse repeats and/or rate-limit.
- Never affects dbt task state. This is observability, not control flow.
- URL always via secret, never logged.

## Follow-up

A shareable blueprint wiring `alertWebhook.url` to a Slack/Teams incoming webhook secret.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.