[C++] Unable to read date64 or date32 in specific format from CSV
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
when importing csv data with dates in the format `"%d-%b-%y"` or `"%d-%b-%Y"` an error is given in conversion:
example:
```python
import pyarrow as pa
from pyarrow import csv
data = b"a,b\n1,15-OCT-15\n2,18-JUN-90\n"
tp = ["%d-%b-%y"]
try:
schema_d64 = pa.schema([pa.field("a", pa.int64()), pa.field("b", pa.date64())])
co_d64 = csv.ConvertOptions(timestamp_parsers=tp, column_types=schema_d64)
a_d64 = csv.read_csv(pa.py_buffer(data), convert_options=co_d64)
except Exception as e:
print(e)
try:
schema_d32 = pa.schema([pa.field("a", pa.int64()), pa.field("b", pa.date32())])
co_d32 = csv.ConvertOptions(timestamp_parsers=tp, column_types=schema_d32)
a_d32 = csv.read_csv(pa.py_buffer(data), convert_options=co_d32)
except Exception as e:
print(e)
```
**Reporter**: [Stephen Bias](https://issues.apache.org/jira/browse/ARROW-12539)
#### Related issues:
- [[C++] Parse time32 from string and infer in CSV reader](https://github.com/apache/arrow/issues/27146) (is related to)
**Note**: *This issue was originally created as [ARROW-12539](https://issues.apache.org/jira/browse/ARROW-12539). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start with the Python reproduction using pyarrow.csv.read_csv, ConvertOptions(timestamp_parsers=...), and date32/date64 column types. Trace how the C++ CSV reader converts the supplied date formats, then verify that both "%d-%b-%y" and "%d-%b-%Y" work for the example data without conversion errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100