apache / apache/ossie

Repo-level cross-converter interop matrix (hub-level testing)

Open
#309 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.1k
Forks
267
Avg merge
4d 20h
Merged PRs (30d)
24

Description

## Problem

Every converter tests its own round trip (`X → Ossie → X`), so all suites can be green while converter *pairs* fail. Real cases found by manually chaining converters during Cube-converter development:

1. **Silently empty output.** A Databricks-authored Ossie document carries only `DATABRICKS`-dialect expressions; an exporter that required `ANSI_SQL` dropped every field and metric and emitted an *empty* target model — exit 0, loadable, nothing downstream noticed.
2. **Mis-attributed aggregates.** Running `Cube → Ossie → Databricks → Ossie → Cube` on the TPC-DS model, `SUM(ss_ext_sales_price)` came back placed on the `customer` cube — a measure over a column that cube does not have. It compiles (SQL is opaque to compilers) and reads the wrong table at query time.
3. **Structural mismatches.** A Databricks metric view cannot carry the source table's own key, so it silently vanishes on the way through; Databricks, Snowflake, and NVIDIA GSF reject the 2-part `source` other converters happily emit. Each converter's own tests pass; the pair fails.

**Prior art:** `converters/cube/tools/interop_matrix.py` — one-directional (`Cube → Ossie → every other spoke`), Cube-owned. This proposal promotes the idea to a repo-level facility owned by everyone.

## Proposal

A repo-level harness (e.g. `interop/`) driven by one small manifest entry per converter — the CLIs genuinely differ today (`export` vs `osi-to-msi` vs no subcommand vs API-only GoodData), and a manifest normalizes over that without forcing a CLI migration first:

```yaml
# interop/manifest.yaml (sketch)
cube:
import: [ossie-cube, import] # native -> Ossie
export: [ossie-cube, export] # Ossie -> native
output: directory
fixture: converters/cube/tests/fixtures/tpcds_cube
toolchain: uv
native_gate: [node, tools/cube_compile.js] # optional, env-gated

databricks:
import: [ossie-databricks, import]
export: [ossie-databricks, export]
output: file
fixture: converters/databricks/tests/fixtures/...
toolchain: uv

polaris:
toolchain: maven # Java converters enter via a Maven runner, or stay SKIP
```

### Test tiers

- **Tier A — Ossie acceptance.** Every `export` must consume the canonical `examples/*.yaml` and every `import`'s output: exit 0 **and non-empty output**.
- **Tier B — pairwise chains.** For each pair (A, B): `A.fixture → Ossie → B.export → B.import → Ossie′`, where `Ossie′` must pass `validation/validate.py` and the JSON Schema.
- **Tier C — native gates.** Where a target has a real compiler/validator (Cube's compiler today, behind `OSSIE_CUBE_REPO`), run it on the chain result.

### Grading

`OK` / `LOSSY (n warns, m foreign-ext warns)` / `EMPTY` / `FAIL` / `SKIP (toolchain)`

- `EMPTY` is a **first-class failure**, not a variant of OK — it is the trap in case 1 above.
- Foreign `custom_extensions` warning counts quantify what each converter's stash costs every other spoke — the number to watch when deciding whether something belongs in an extension at all.

### CI

Report-only markdown matrix as a build artifact first. Once green, ratchet Tier A on `examples/tpcds_semantic_model.yaml` to blocking. Java converters (Polaris, Salesforce) stay `SKIP` until a Maven runner is wired.

## Offer

We will contribute the harness by generalizing the existing Cube tool; each converter's owners only fill in their manifest entry.

Sent discussion: `[DISCUSS] Repo-level cross-converter interop matrix` on `dev@ossie.apache.org` but they don't appear on lists.apache.org for some reasons =(

Contributor guide

Open the contributing guide

Research direction

Start with converters/cube/tools/interop_matrix.py to understand the existing one-directional matrix, then review validation/validate.py and the JSON Schema. Use the proposed interop/manifest.yaml and examples/tpcds_semantic_model.yaml as the stated entry points. Done means a report-only matrix can grade the described tiers and distinguish OK, LOSSY, EMPTY, FAIL, and SKIP.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.