apache / apache/dolphinscheduler
[DSIP-79][Task] Add Datavines task to better support data quality
- Dominant language
- Java
- Stars
- 14.5k
- Forks
- 5.1k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 28
Description
### Search before asking
- [X] I had searched in the [DSIP](https://github.com/apache/dolphinscheduler/issues/14102) and found no similar DSIP.
### Motivation
DataVines is an easy-to-use data quality service platform that supports multiple metric.
https://github.com/datavane/datavines
- Datavines supports executing multiple metrics in one job.
- Datavines supports execution status dashboard and data quality report.
- Datavines supports plug-in extensions for components such as metric, data sources, error data storage, and execution engines.
- Jdbc engines can be used to execute data quality tasks instead of solely relying on Spark engines.
### Design Detail
#### Sript mode
1. config data quality job in datavines

2. get the job config scipt file
3. Add datavines job node in workflow, and configure the script

#### API Mode
1. config data quality job in datavines

2. get the jobId
3. Add datavines job node in workflow, and configure the datavines api address and jobId
### Compatibility, Deprecation, and Migration Plan
_No response_
### Test Plan
_No response_
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Assessment
This issue has not been assessed yet.