cdisc-org / cdisc-org/DataExchange-DatasetJson

Clarification on null and empty string handling for decimal and string types

Open
#46 3 comments 1 reaction 0 assignees View on GitHub
More information needed User's Guide
Dominant language
Jupyter Notebook
Stars
41
Forks
3
PR merge metrics
No merged PRs in 30d

Description

While writing type-aware conversion and validation code for the `rows` element, I noticed a couple of points that I would like to clarify.

For decimal values, when the value is missing, examples such as `adlbh` use empty string.
In the specification, missing values are defined as follows:

> Missing values are represented by null. Empty strings are represented by "".

However, the handling of decimal values is not entirely clear.
In my opinion, null would be more appropriate for a missing decimal value, but is it intended that empty strings are also acceptable?

While checking the implementation examples related to this, I also noticed that empty string are generally used for missing string values.
This may reflect the SAS convention where an empty string represents a missing value for character variables. However, in many modern systems, null and empty string are clearly distinguished, and treating empty strings simply as missing values could potentially cause issues.

Possible approaches might include:

• Explicitly define that empty string and null are not distinguished and should be treated equivalently.
• Add a column attribute to specify which values should be treated as missing (e.g., missing="").
• Add a column attribute indicating that empty strings should be preserved separately from missing values (e.g., keep_empty=true).

If there is an intended way to handle this, it would be very helpful to know.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.