databrickslabs / databrickslabs/ontos
[Feature]: Compliance: SCHEMA_DRIFT check function
Open
Nobody has claimed this yet.
type/feature
- Dominant language
- Python
- Stars
- 212
- Forks
- 71
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 43
Description
Parent PRD
#125
What to build
Implement the SCHEMA_DRIFT(physical_name) DSL check function end-to-end:
- Uses
EvaluationContext.workspace_clientto run DESCRIBE TABLE on the physical UC table - Compares UC columns/types against contract schema properties from the enriched entity dict (requires
schemafacet) - Detects: missing columns (in contract but not in table), extra columns (in table but not in contract), type-changed columns
- Returns CheckResult with pass/fail, human-readable message, and details_json containing the drift specifics
- Reuses logic from the existing
data_contract_validation.pycheck_schema_driftfunction - Register in the check function registry
- Tests with mocked WorkspaceClient covering: no drift, missing columns, extra columns, type changes, empty schema
Acceptance criteria
-
SCHEMA_DRIFTfunction registered in check function registry - Compares contract schema properties against UC DESCRIBE TABLE output
- Correctly identifies missing, extra, and type-changed columns
- Returns structured details_json with drift specifics
- Tests with mocked WorkspaceClient cover all drift scenarios
Blocked by
- Blocked by #127 (EvaluationContext + check function registry)
- Blocked by #128 (facet-based entity enrichment for schema facet)
User stories addressed
- User story 9
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
This work depends on #127 and #128; start with the EvaluationContext and check function registry entry points, then review the existing check_schema_drift function in data_contract_validation.py. Use mocked WorkspaceClient tests for no drift, missing columns, extra columns, type changes, and an empty schema; done means the registered SCHEMA_DRIFT function returns structured drift details and correct pass/fail results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- backend, data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100