dbt-labs / dbt-labs/metricflow

[Feature] Support 2-step aggregation for metrics

Open
#653 6 comments 1 reaction 1 assignee Claimed by @Jstein77 View on GitHub
backlog enhancement
Dominant language
Python
Stars
1.8k
Forks
202
Avg merge
1d 8h
Merged PRs (30d)
14

Description

### Is this your first time submitting a feature request?

- [X] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing metricflow functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

As we discussed [on Slack](https://getdbt.slack.com/archives/C02CCBBBR1D/p1689036406589729), we figured out derived metrics doesn't support to aggregate metrics in derived metrics. So, it would get better to support to aggregating operations in derived metrics.

We consider if we calculate the number of users by the number of page views who the users access for a certain duration from access logs. First, we calculate the number of page views per user. Then we calculate the number of users per the calculated page views per user.

```sql
WITH page_views_by_user (
SELECT
user_id,
COUNT(*) AS page_views
FROM accesslog
WHERE DATE(event_time) >= DATE("2023-01-01") -- The duration can be parameterized
GROUP BY 1
)

SELECT
page_views,
COUNT(*) AS num_users
FROM page_views_by_user
GROUP BY 1
ORDER BY 1

------
page_views,num_users
1,xxx
2,xxx
3,xxx
4,xxx
```

### Describe alternatives you've considered

We can create another dbt model to calculate the number of page views per user right now. But if we want to change the duration, that might be a bit harder. That’s because we have to calculate the number of page views per users and a certain duration say about day a head. Then, we can sum up them by user to calculate the total number of pages views in the desired duration.

```csv
user,date,page_views
1,2023-01-01,2
1,2023-01-02,1
...
```

And there are other use cases that the precedent approach with creating another model doesn’t work. We assume if we want to calculate the total number of types of items per user on an e-commerce service. We have to count distinct items per user. However, if we want to dynamically change the duration with a condition, the preceding approach doesn’t work. We can’t get the distinct value per user by summing up the data over a finer period of time say about day. So, I think it would be worthwhile to support aggregations in the derived metrics in the future.

```sql
WITH unique_items_per_user (
SELECT
user_id,
COUNT(DISTINCT item_id) AS num_unique_items
FROM orders
WHERE DATE(purchased_at) >= DATE("2023-01-01") -- The duration can be parameterized
GROUP BY 1
)

SELECT
num_unique_items,
COUNT(user_id) AS num_users
FROM page_views_by_user
GROUP BY 1
ORDER BY 1

------
num_unique_items,num_users
1,xxx
2,xxx
3,xxx
4,xxx
```

### Who will this benefit?

If we support the feature, the feature can enable data analysts who want to flexibly calculate metrics over metrics without creating other models. And as I described above, there are some use cases which the preceding approach doesn't work to get distinctive values. The feature can solves the two main use cases.

### Are you interested in contributing this feature?

yes

### Anything else?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.