databrickslabs / databrickslabs/ontos
[Feature]: Compliance: per-entity scheduling + Databricks job refactor
Open
Nobody has claimed this yet.
type/feature
- Dominant language
- Python
- Stars
- 212
- Forks
- 71
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 43
Description
Parent PRD
#125
What to build
Implement per-entity schedule overrides and refactor the Databricks compliance job:
- Per-entity scheduling: The
schedulefield on entity_compliance_checks supports:daily,weekly,on_change,manual. - on_change trigger: Wire into entity update lifecycle hooks (on_update for contracts/products). When an entity with
on_changebindings is updated, triggerrun_entity_checkfor those bindings immediately. - ComplianceManager.run_all_entity_checks(db, workspace_client, schedule_filter): Iterate all active bindings matching the schedule filter, run each. Respects
last_run_atto avoid re-running within the schedule window. - Job refactor: Add
--modeparameter tocompliance_checks.py:policies(existing sweep behavior) orentity_checks(callsrun_all_entity_checks). Job can run both modes in sequence. - Deprecate: Mark
data_contract_validation.pyjob as deprecated (its logic is now in DSL check functions). Remove or keep as no-op with deprecation warning.
Acceptance criteria
- Schedule field respected: daily/weekly/on_change/manual
- on_change bindings triggered on entity update via lifecycle hooks
- run_all_entity_checks iterates active bindings for the given schedule
- Databricks job supports --mode=policies and --mode=entity_checks
- data_contract_validation.py deprecated
- last_run_at checked to avoid redundant runs within schedule window
Blocked by
- Blocked by #126 (entity_compliance_checks data layer)
- Blocked by #134 (lifecycle hooks)
- Blocked by #138 (run_entity_check execution engine)
User stories addressed
- User story 6
- User story 24
- User story 25
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing blockers #126, #134, and #138, then inspect compliance_checks.py and data_contract_validation.py. Trace the entity update lifecycle and ComplianceManager entry points mentioned in the issue. Done means all listed schedule behaviors, mode handling, last_run_at checks, and deprecation behavior satisfy the acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data-engineering, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100