ckan / ckan/ideas

Data Enrichment Extensions

Open
#151 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
39
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Once #150 is implemented and we have a more robust way to upload data into the Datastore, we'll have the necessary metadata (JSON Table Schema) to support the development of Data Enrichment Extensions.

Perhaps, enrichment can be done by either by local services or remote services using webhooks (#122)?

Some obvious data-enrichment services:
- Geocoding (with plugins for various geocoding services, perhaps, leveraging [geopy](https://github.com/geopy/geopy)). In NYC, we already developed a [Geoclient](https://developer.cityofnewyork.us/api/geoclient-api) geocoder for geopy. :smile:
- Wikidata reconciliation
e.g. https://www.wikidata.org/w/api.php?action=wbsearchentities&search=CKAN&language=en&type=item
- Opencorporates API
- [CitySDK](http://uscensusbureau.github.io/citysdk/)
- tighter OpenRefine integration - i.e. clearer association/presentation of pre-refined dataset, recipe, and refined dataset

Data Enrichment Extensions are a special class of CKAN extensions that operate on Datastore data.

To obviate the need for revisions, I suggest that all enrichment will only be additive - adding columns perhaps with a special prefix (e.g. _e_latitude, _e_longitude, etc.) This is similar to how CartoDB adds columns to data when you import it (i.e. cartodb_id, the_geom).

Having this convention to enriched column names may actually help with linking entities across datasets (e.g. _e_wikidata_id, _e_wikidata_url) as the names are standardized.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.