dbt-labs / dbt-labs/metricflow

[Feature] Approximate count distinct

Open
#1,682 1 comment 0 reactions 0 assignees View on GitHub
enhancement triage
Dominant language
Python
Stars
1.8k
Forks
202
Avg merge
1d 8h
Merged PRs (30d)
14

Description

### Is this your first time submitting a feature request?

- [x] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [x] I have searched the existing issues, and I could not find an existing issue for this feature
- [x] I am requesting a straightforward extension of existing metricflow functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

There are cases where a count distinct is too computationally heavy, resulting in slow queries and high costs. Approximate counts can be much faster while still being accurate. Example measure:

```yaml
name: unique_events
description: Unique Events
expr: event_id
agg: count_distinct
agg_params:
use_approximate: True
```

### Describe alternatives you've considered

Plain count distincts aren't an option due to performance. When using semantic models built on top of pre aggregated models, we will lose too much flexibility.

### Who will this benefit?

_No response_

### Are you interested in contributing this feature?

Yes

### Anything else?

Approx percentiles already exists in metricflow:
https://github.com/dbt-labs/metricflow/pull/395

Similar measure in Looker:
```yaml
measure: apx_unique_count {
type: count_distinct
approximate: yes # default value is no
sql: ${id} ;;
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.