ckan / ckan/datapusher

datapusher appending rows to existing datastore

Open
#98 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
82
Forks
157
PR merge metrics
No merged PRs in 30d

Description

I'm updating a few resources nightly using the python ckanapi resource_update action. The resource is updating in the filestore but it we're seeing duplicate rows in the datastore, so if the resource has 1000 rows we often see 2000 in the data explorer and when using the datastore api. When we manually click the 'Upload to Datastore' button, we do not encounter this error.

In the datapusher jobs.py file there is reference to this error:
'''
Delete existing datstore resource before proceeding. Otherwise
'datastore_create' will append to the existing datastore. And if
the fields have significantly changed, it may also fail.
'''

What is the difference in terms of the datastore usage between manually and automatically updating a resource? Is there a workaround for automatically updating the datastore?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.