dbt osi-to-msi silently corrupts composite primary and unique keys
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 267
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 24
Description
## Problem
The dbt `OSIToMSIConverter` flattens every Ossie key group into a set of columns. For a composite primary key such as `[tenant_id, order_id]`, it emits two independent MetricFlow `PRIMARY` entities. The reverse conversion then keeps only the final primary entity, silently changing the key to `[order_id]`. Composite unique keys are likewise converted into independent singleton unique keys.
## Reproduction
On `main` at `88e0011148283302c9a04cd0287e00e0b9d87354`, convert a valid dataset containing:
```yaml
primary_key: [tenant_id, order_id]
```
Observed after Ossie -> MSI:
```text
entities=[(tenant_id, primary), (order_id, primary)]
```
Observed after Ossie -> MSI -> Ossie:
```yaml
primary_key: [order_id]
```
A singleton primary key round-trips unchanged.
## Root cause
`converters/dbt/src/ossie_dbt/osi_to_msi.py::_build_key_sets` converts `primary_key` to a set and flattens all `unique_keys` groups. `_classify_field` then treats every member as independently primary or unique. `MSIToOSIConverter._extract_keys` has no grouping information to reconstruct.
## Impact
Composite keys are common for tenant-scoped and line-item datasets. Emitting each component as independently unique can produce incorrect join cardinality or aggregations, while round trips silently lose key components.
## Proposed fix
Until MSI can represent and round-trip grouped keys losslessly, fail Ossie -> MSI conversion clearly when a dataset contains a composite primary or unique key. Keep singleton key behavior unchanged and add regression tests for both composite key types. This prevents generating a plausible but semantically incorrect manifest.
Contributor guide
Research direction
Start in converters/dbt/src/ossie_dbt/osi_to_msi.py, reading _build_key_sets and _classify_field, then inspect MSIToOSIConverter._extract_keys to understand the missing grouping information. Add regression coverage for composite primary and unique keys while preserving singleton behavior. Done means composite keys fail clearly during Ossie-to-MSI conversion instead of producing misleading independent entities.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100