Repo-level cross-converter interop matrix (hub-level testing)
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 267
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 24
Description
## Problem
Every converter tests its own round trip (`X → Ossie → X`), so all suites can be green while converter *pairs* fail. Real cases found by manually chaining converters during Cube-converter development:
1. **Silently empty output.** A Databricks-authored Ossie document carries only `DATABRICKS`-dialect expressions; an exporter that required `ANSI_SQL` dropped every field and metric and emitted an *empty* target model — exit 0, loadable, nothing downstream noticed.
2. **Mis-attributed aggregates.** Running `Cube → Ossie → Databricks → Ossie → Cube` on the TPC-DS model, `SUM(ss_ext_sales_price)` came back placed on the `customer` cube — a measure over a column that cube does not have. It compiles (SQL is opaque to compilers) and reads the wrong table at query time.
3. **Structural mismatches.** A Databricks metric view cannot carry the source table's own key, so it silently vanishes on the way through; Databricks, Snowflake, and NVIDIA GSF reject the 2-part `source` other converters happily emit. Each converter's own tests pass; the pair fails.
**Prior art:** `converters/cube/tools/interop_matrix.py` — one-directional (`Cube → Ossie → every other spoke`), Cube-owned. This proposal promotes the idea to a repo-level facility owned by everyone.
## Proposal
A repo-level harness (e.g. `interop/`) driven by one small manifest entry per converter — the CLIs genuinely differ today (`export` vs `osi-to-msi` vs no subcommand vs API-only GoodData), and a manifest normalizes over that without forcing a CLI migration first:
```yaml
# interop/manifest.yaml (sketch)
cube:
import: [ossie-cube, import] # native -> Ossie
export: [ossie-cube, export] # Ossie -> native
output: directory
fixture: converters/cube/tests/fixtures/tpcds_cube
toolchain: uv
native_gate: [node, tools/cube_compile.js] # optional, env-gated
databricks:
import: [ossie-databricks, import]
export: [ossie-databricks, export]
output: file
fixture: converters/databricks/tests/fixtures/...
toolchain: uv
polaris:
toolchain: maven # Java converters enter via a Maven runner, or stay SKIP
```
### Test tiers
- **Tier A — Ossie acceptance.** Every `export` must consume the canonical `examples/*.yaml` and every `import`'s output: exit 0 **and non-empty output**.
- **Tier B — pairwise chains.** For each pair (A, B): `A.fixture → Ossie → B.export → B.import → Ossie′`, where `Ossie′` must pass `validation/validate.py` and the JSON Schema.
- **Tier C — native gates.** Where a target has a real compiler/validator (Cube's compiler today, behind `OSSIE_CUBE_REPO`), run it on the chain result.
### Grading
`OK` / `LOSSY (n warns, m foreign-ext warns)` / `EMPTY` / `FAIL` / `SKIP (toolchain)`
- `EMPTY` is a **first-class failure**, not a variant of OK — it is the trap in case 1 above.
- Foreign `custom_extensions` warning counts quantify what each converter's stash costs every other spoke — the number to watch when deciding whether something belongs in an extension at all.
### CI
Report-only markdown matrix as a build artifact first. Once green, ratchet Tier A on `examples/tpcds_semantic_model.yaml` to blocking. Java converters (Polaris, Salesforce) stay `SKIP` until a Maven runner is wired.
## Offer
We will contribute the harness by generalizing the existing Cube tool; each converter's owners only fill in their manifest entry.
Sent discussion: `[DISCUSS] Repo-level cross-converter interop matrix` on `dev@ossie.apache.org` but they don't appear on lists.apache.org for some reasons =(
Contributor guide
Research direction
Start with converters/cube/tools/interop_matrix.py to understand the existing one-directional matrix, then review validation/validate.py and the JSON Schema. Use the proposed interop/manifest.yaml and examples/tpcds_semantic_model.yaml as the stated entry points. Done means a report-only matrix can grade the described tiers and distinguish OK, LOSSY, EMPTY, FAIL, and SKIP.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100