dbt-labs / dbt-labs/metricflow

OSI converter: composite primary_key cannot produce a valid semantic model (multiple PRIMARY entities vs missing primary entity)

Open
#2,093 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.8k
Forks
202
Avg merge
1d 8h
Merged PRs (30d)
14

Description

## Bug: OSI datasets with a composite primary_key cannot produce a valid semantic model

The OSI spec explicitly supports composite primary keys (`primary_key: [order_id, line_number]`), but `OSIToMSIConverter` maps **each** primary-key column that is also declared as a field to its own `EntityType.PRIMARY` entity:

```python
# metricflow/converters/osi_to_msi.py
primary=set(dataset.primary_key or []),
...
if field.name in primary_key_cols:
... type=EntityType.PRIMARY ...
```

MetricFlow validation only allows one primary entity per semantic model, so composite keys are stuck in a catch-22 (observed via dbt-core 1.12.0 native OSI parsing, bundled metricflow 0.211.0):

- Declare the composite key columns as fields →
`Semantic models can have only one primary entity. The semantic model 'orders' has 2: order_id, line_number` → `Semantic Manifest validation failed.`
- Omit `primary_key` (the dataset has dimensions) →
`The semantic model orders contains dimensions, but it does not define a primary entity. Either add an entity with type PRIMARY or set a value for the primary_entity key.` → `Semantic Manifest validation failed.`

The only shape that parses today is a single-column `primary_key` — i.e. any dataset whose true grain is composite either fails validation or has to under-declare its fields so the extra key columns are silently ignored.

### Steps to reproduce

Dataset with `primary_key: ["order_id", "line_number"]` and both columns declared in `fields`, parsed via dbt-core 1.12 (`osi/` directory). Fails with the "only one primary entity" message above. Removing `primary_key` fails with the "does not define a primary entity" message.

### Possible directions

- Map a composite `primary_key` to a single derived entity (e.g. the model's `primary_entity` name with the key columns as its expr components), or
- Map composite keys to `EntityType.UNIQUE` / a virtual `primary_entity` so validation passes while preserving the declared grain.

Related: #2091 (ratio sub-metric naming) — both found while testing dbt 1.12 native OSI parsing against a production dbt project.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.