apache / apache/ossie

Core spec: root-level `dialects`/`vendors` are allowed by the schema since #297 but treated as non-spec everywhere else

Closed
#393 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.1k
Forks
267
Avg merge
4d 20h
Merged PRs (30d)
24

Description

## Problem

Since #297 (merged 2026-09-08) the document root of `core-spec/ossie-schema.json` accepts two optional arrays, `dialects` and `vendors`. The PR describes this as closing a gap: the pydantic `OssieDocument` already had both fields, the schema did not.

The rest of the repository had settled the same mismatch the other way round, taking the schema as canonical:

- #148 (open, approved in June, waiting on a rename conflict) removes both fields from `OssieDocument` and stops dbt from emitting a root `dialects`. Its premise, that dbt output fails `validate.py` with `Additional properties are not allowed ('dialects' was unexpected)`, no longer holds on main.
- #306 (open) does the same for dbt and Honeydew.
- `converters/orionbelt/tests/test_ossie_v02_compat.py::test_schema_rejects_root_dialects_and_vendors` asserts, citing #148, that the schema rejects both. It fails on main; the orionbelt workflow only runs on `converters/orionbelt/**` and last ran the day before #297 merged, so the next PR touching that directory gets an unrelated red build.
- `core-spec/spec.md` does not mention either field at the document root. The `dialects:` at the top of `spec.yaml` is the enumeration of dialect names, not a document field.

A document with `dialects: [ANSI_SQL]` and `vendors: [DBT]` at the root passes `validate.py` on main and fails against the schema as of the commit before #297.

Beyond the schema descriptions the two fields have no semantics: nothing says whether the lists must be complete, whether using an unlisted dialect makes a document invalid, or what a consumer may do with them, and `validate.py` does not read them.

## Proposal

Take the two properties back out of the schema until they have a definition: revert the two root properties from #297, have the Sigma converter stop emitting them (dbt and Honeydew are covered by #148 and #306), and let #148 and #306 proceed; the orionbelt test is then right as it stands. It is the smaller change, it matches the prose spec, and nothing consumes the lists today: dbt and Sigma write them, Honeydew only carries them through.

If the index is wanted, the definition should come first: a document-level section in `spec.md` saying what the lists mean (an advisory summary, or a constraint the validator checks), with the schema, the orionbelt test and #148/#306 following from it. Either way it is worth settling before #383 moves where "root" is. I can send the follow-up PR for whichever direction is taken.

Contributor guide

Open the contributing guide

Research direction

Start by comparing core-spec/ossie-schema.json with core-spec/spec.md, then run validate.py and converters/orionbelt/tests/test_ossie_v02_compat.py::test_schema_rejects_root_dialects_and_vendors. Trace the Sigma converter and the related dbt and Honeydew issues (#148 and #306). Done means the schema, converter output, validator behavior, and compatibility test agree, or the specification defines the fields and all affected parts follow it.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, testing
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.