ckan / ckan/datapusher

KeyError with messytables.DateType

Open
#96 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
82
Forks
157
PR merge metrics
No merged PRs in 30d

Description

Hi all,

I have Datapusher configured with the following type configuration:

```
TYPES = [messytables.StringType, messytables.DecimalType, messytables.IntegerType, messytables.DateType, messytables.BoolType]
TYPE_MAPPING = {
'String': 'text',
# 'int' may not be big enough,
# and type detection may not realize it needs to be big
'Integer': 'numeric',
'Decimal': 'numeric',
'Date': 'timestamp',
'Bool': 'boolean'
}
```

When a column's type is determined to be a date, the string representation from `messytables.DateType` includes the matched format, for example: `Date(%d/%m/%Y)`. As we don't have `Date(%d/%m/%Y)` in `TYPE_MAPPING` datapusher will throw a KeyError exception.

I've worked around this for now, by inserting the following into my `datapusher_settings.py`, though it feels rather hacky:

```
from messytables.dateparser import DATE_FORMATS

# ...

# Include all DateType representable formats.
# The matched types can be described as `Date(%d/%m/%Y)`, so we need
# to have a mapping for every format available.
for format in DATE_FORMATS:
TYPE_MAPPING['Date({})'.format(format)] = 'timestamp'
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.