dbt-labs / dbt-labs/dbt-agent-skills

Answering natural language questions skill doesn't cause the agent to check for a more appropriate metric when the request changes

Open
#64 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
710
Forks
62
Avg merge
1d 18h
Merged PRs (30d)
5

Description

## Which skill is affected?

answering-natural-language-questions-with-dbt

## What happened?

### The skill triggered but didn't behave as expected

**What did you expect?**

When choosing the wrong metric on the first run through, it should check for a more appropriate metric instead of just manually doing maths on the original dataset.

**What happened instead?**

```
❯ graph customer count and arr increase by month

⏺ I'll use the dbt Semantic Layer to query the data and then graph it. Let me start by loading the relevant skills and checking what metrics are available.

⏺ Skill(dbt:answering-natural-language-questions-with-dbt)
⎿  Successfully loaded skill

⏺ Using the dbt Semantic Layer to answer this question. Let me check what metrics are available.

...

The chart should now be open in your browser. It shows a combo chart with:

- Bars: Customer count (new customers per month) on the right axis
- Line: ARR growth on the left axis

ARR grew steadily from ~$6K in Jan 2024 to ~$135K by late 2025, while new customer additions tapered off after mid-2024 — the later ARR growth appears driven by
expansion revenue from existing customers rather than new logos.
```

```
❯ show cumulative customer count, not net new, and add the semantic layer query at the bottom of the chart so it's replicable later if needed

⏺ Done — the chart now has a text block at the bottom showing the semantic layer query details (metrics, group by, order by) and a note that cumulative values were
derived from the net new counts

❯ can the semantic layer not do running total natively?

⏺ Good question. MetricFlow does support cumulative metric types natively. Let me check — there's an active_customers metric that might already be cumulative.
⏺ Yes — active_customers is already a cumulative metric. Those numbers are close to what I manually computed (slight differences likely due to how the metric handles the
accumulation window). Let me re-query the full dataset and update the chart.
⏺ Even better — active_customers is a proper cumulative metric that also accounts for churn (peaks at 69, not 75). Let me update the chart with the correct data.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.