[Feature]: Integrate DQX for DQ checks of Data Contracts

Is there an existing issue for this?

• I have searched the existing issues

Problem statement

Contracts can define DQ rules, they have to be implement somehow.

Proposed Solution

Use DQX to implement the DQ rules and surface the findings via the Ontos UI. The Compliance service defines one or more compliance check for every active contract. These are run as jobs on a Databricks job cluster (configured in the settings), and have a UI to view all recent job runs and the results of them.

DQX supports summary tables that can be used to show results for large operations (think 1T row scan) instead of listing all violations separately. For up to certain number of rows the option is to show both, the summary and the individual violations.

Additional Context

No response

Start by mapping the Compliance service, DQX integration, Databricks job-cluster configuration, and Ontos UI described in the proposal. Define how checks run for each active contract, how job runs and results are displayed, and when summary versus individual violations are shown. Done means the end-to-end design and implementation support these workflows.