DataStore column name mapping
- Dominant language
- No language data
- Stars
- 39
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
When importing a CSV into DataStore, the column names are set (by DataPusher/[Express Loader](https://github.com/davidread/ckanext-xloader)) to be the same in the database as they are in the CSV.
This is problematic:
* [In queries its confusing to have to double quote most column names](https://github.com/ckan/ckan/issues/2577)
* Postgres doesn't allow unicode characters. So DataPusher/Express Loader can't upload them.
* Postgres limit on column names length is 63 characters
* Repeated column names
How about we get DataPusher/Express Loader to make some sensible changes to column names before it goes into the database, and then the mapping is saved for:
* users can see it in the web interface
* users can get it via an API
* users can *edit* it in the web interface
* subsequent DataPusher/Express Loader runs reuse the same mapping if possible
Contributor guide
No contributing guide indexed for this repository
Research direction
Review the DataStore import behavior and the DataPusher/Express Loader integration first. Define the scope for normalizing CSV column names and persisting their mapping. Done means the mapping handles PostgreSQL restrictions and duplicate names, is visible and editable in the web interface, available through an API, and reusable on later imports.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql
- Domain
- api, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100