inveniosoftware / inveniosoftware/dojson
filter away empty fields/subfields after input
- Dominant language
- Python
- Stars
- 11
- Forks
- 30
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
Currently, `utils.filter_values()` is filtering away keys and corresponding values from dictionaries where `value is None`.
This concretely means, e.g. in the context of MARC21 conversion to JSON, that subfields with empty strings would be preserved, datafields with no subfields would be preserved.
## Proposal
If we assume that an empty string _in the bibliographic metadata context_ doesn't carry any valuable information, it is proposed that `filter_values` actually filters away any key whose value is:
- evaluate to `False`
- unless it's the `False` value itself (thus representing flag set to false) or the 0 number
## Usecases
According to TIND, @Kennethhole reports:
> I can confirm that TIND does not intend to use empty fields. However, it is highly likely that there are empty subfields in our databases and we prefer that dojson don't break due to that! From our point of view, these subfields can be removed during the conversion.
Related to INSPIRE, I can confirm that we have no use for empty values and we internally went further and have implemented a function that recursive visit the whole record and strips away also empty list and empty dicts that result from having filtered values.
https://github.com/inspirehep/inspire-next/blob/master/inspirehep/dojson/utils/__init__.py#L206
See also:
- https://github.com/inveniosoftware/invenio/issues/1654
- https://github.com/inveniosoftware/dojson/pull/155
Contributor guide
Assessment
This issue has not been assessed yet.