databrickslabs / databrickslabs/ontos

[Feature]: Compliance: per-entity scheduling + Databricks job refactor

Open
#142 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/feature
Dominant language
Python
Stars
212
Forks
71
Avg merge
4d 10h
Merged PRs (30d)
43

Description

Parent PRD

#125

What to build

Implement per-entity schedule overrides and refactor the Databricks compliance job:

  • Per-entity scheduling: The schedule field on entity_compliance_checks supports: daily, weekly, on_change, manual.
  • on_change trigger: Wire into entity update lifecycle hooks (on_update for contracts/products). When an entity with on_change bindings is updated, trigger run_entity_check for those bindings immediately.
  • ComplianceManager.run_all_entity_checks(db, workspace_client, schedule_filter): Iterate all active bindings matching the schedule filter, run each. Respects last_run_at to avoid re-running within the schedule window.
  • Job refactor: Add --mode parameter to compliance_checks.py: policies (existing sweep behavior) or entity_checks (calls run_all_entity_checks). Job can run both modes in sequence.
  • Deprecate: Mark data_contract_validation.py job as deprecated (its logic is now in DSL check functions). Remove or keep as no-op with deprecation warning.

Acceptance criteria

  • Schedule field respected: daily/weekly/on_change/manual
  • on_change bindings triggered on entity update via lifecycle hooks
  • run_all_entity_checks iterates active bindings for the given schedule
  • Databricks job supports --mode=policies and --mode=entity_checks
  • data_contract_validation.py deprecated
  • last_run_at checked to avoid redundant runs within schedule window

Blocked by

  • Blocked by #126 (entity_compliance_checks data layer)
  • Blocked by #134 (lifecycle hooks)
  • Blocked by #138 (run_entity_check execution engine)

User stories addressed

  • User story 6
  • User story 24
  • User story 25

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing blockers #126, #134, and #138, then inspect compliance_checks.py and data_contract_validation.py. Trace the entity update lifecycle and ComplianceManager entry points mentioned in the issue. Done means all listed schedule behaviors, mode handling, last_run_at checks, and deprecation behavior satisfy the acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering, devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.