h2oai / h2oai/datatable

Add parameter `na_string=` to .to_csv()

Open
#2,166 4 comments 0 reactions 1 assignee Claimed by @Mathanraj-Sharma View on GitHub
Beginner task improve
Dominant language
C++
Stars
1.9k
Forks
164
Avg merge
7h 31m
Merged PRs (30d)
1

Description

I encountered an unexpected behavior of `to_csv()` and `fread()` regarding its handling of 'NA' string.

When I ran the following code,

```python
import datatable as dt

data = dt.Frame(['a', 'NA'])
print(data[1,0], type(data[1,0]))
data.to_csv("NA-test.csv")
data = dt.fread("NA-test.csv")
print(data[1,0], type(data[1,0]))
```
I had expected that I would get the same string 'NA <class str>' and 'NA <class str>' from both print statements, because, after all, all I did was just to serialize data to a file and read it back. Unfortunately, the actual result is 'NA <class str>' and 'None <class NoneType>'.

The current behavior is "understandable" given that 'NA' is one of the default strings to represent an NA value. But the combined default behaviors of `to_csv()` and `fread()` led to a surprise that I did not expect at all.

I can work around this problem by adding the parameter `na_strings=[""]` to `fread()` like the following:

```python
import datatable as dt

data = dt.Frame(['a', 'NA'])
print(data[1,0], type(data[1,0]))
data.to_csv("NA-test.csv")
data = dt.fread("NA-test.csv", na_strings=[""])
print(data[1,0], type(data[1,0]))
```

But this workaround won't work if I have a true NA value in another column.

Will it be possible to "escape" the output of the string 'NA' in `to_csv()` by default somehow, so that when it is read back, it remains to be the string 'NA', not the NA value?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.