ckan / ckan/ckanext-validation

Validate on dataset creation

Open
#11 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
30
Forks
33
PR merge metrics
No merged PRs in 30d

Description

What will really make a difference in terms of improving data quality will be the integration of the validation process in the process for adding new datasets.

Running the validation before the creation of the dataset will highlight errors early in process and engage publishers in fixing those issues.

The actual workflow will depend on each CKAN instance and the publishers needs, for instance some might prevent the dataset from being created if there are errors, or allow the creation of the dataset but not upload it to the DataStore, or in some case cases just warn the user.

Regardless of how the process continues after the validation, the main implementation issue would be how to integrate an asynchronous validation job with the multi-step process now in place for creating datasets. The resource endpoints of the `package` controller are infamous for their complexity, caused by how this multi-stage process is implemented.

*Estimate (see comment below): 7 days*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by examining the resource endpoints of the package controller and the existing multi-step dataset-creation workflow. Trace how validation currently runs and how an asynchronous job could report its result before creation proceeds. Done means the workflow integrates validation and supports the instance-specific behavior described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.