Spec has no way to declare a non-additive metric
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 267
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 24
Description
## The gap
Ossie can't say that a metric is non-additive along a dimension. Metrics are static SQL strings, and the spec doesn't define the join semantics they run under.
So two conforming consumers can get different numbers from the same model.
## Example
```yaml
datasets:
- name: orders # many
primary_key: [id]
- name: customers # one
primary_key: [id]
relationships:
- name: orders_to_customers
from: orders
to: customers
from_columns: [customer_id]
to_columns: [id]
metrics:
- name: total_lifetime_value
expression:
dialects:
- dialect: ANSI_SQL
expression: SUM(customers.lifetime_value)
```
Group `total_lifetime_value` by `orders.status`. The join repeats each customer row once per order, so the `SUM` over-counts by the customer's order count.
The model isn't wrong. The spec just doesn't say which answer is correct.
## Two converters already hit this
**dbt.** MetricFlow has `non_additive_dimension`. Ossie has no field for it, so `converters/dbt/src/ossie_dbt/osi_to_msi.py` sets `non_additive_dimension=None` on export, and import records a `CUMULATIVE_SEMANTICS_LOSS` issue.
**Cube.** Cube fixes this at query time. When a cube is on the multiplied side of a join, it selects the distinct primary keys, joins them back to the measure's own cube, and aggregates there. Each row counts once. It refuses the query when the measures themselves span cubes that fan out. That's a runtime rewrite driven by declared primary keys, so a static expression can't inherit it.
That makes Cube → Ossie → Databricks unsafe for these metrics. A Databricks metric view has join `cardinality` and `rely.at_most_one_match`, but `at_most_one_match` asserts there is no fan-out. It isn't a dedup instruction. A `SUM` over a `one_to_many` join is computed on the flattened rows.
## Options
Ossie already has enough to detect this: `primary_key` plus directional `from`(many)/`to`(one) relationships. What's missing is a way to declare it.
1. **Spec language only.** Require consumers to deduplicate on the aggregated dataset's `primary_key` when a relationship fans it out. No schema change, but it changes what existing models mean.
2. **A metric-level field.** `non_additive_dimension` (following dbt), or an `additivity` / `grain` marker.
3. **Leave it to `custom_extensions`** and accept these metrics don't round-trip portably.
Is this worth pursuing, and which direction? I can write up a concrete proposal for whichever the list prefers.
## Context
Came up building the Ossie ↔ Cube converter (#289), where it's the biggest fidelity issue. Possibly adjacent to #279, though that's about dialect drift rather than join semantics.
Contributor guide
Research direction
Start by reading the existing schema definitions for primary_key and directional relationships, then inspect converters/dbt/src/ossie_dbt/osi_to_msi.py and the related import path. Compare the three proposed approaches with the dbt and Cube behavior described in the issue. Done means a maintainer-approved concrete proposal that defines the spec change, compatibility impact, and converter handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql, yaml
- Domain
- data, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100