[C++] Writing a dataset with 0 rows doesn't create any files
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
If the input data has no rows, no files get created. This is potentially unexpected as it looks like "nothing happened". It might be nicer to create an empty file. With partitioning, though, that then gets weird (there's no partition values) so maybe an error might make more sense instead.
Reproduction in Python
```python
import tempfile
from pathlib import Path
import pyarrow
import pyarrow.dataset
print("PyArrow version:", pyarrow.__version__)
table = pyarrow.table([
[],
], schema=pyarrow.schema([
("ints", "int64"),
]))
with tempfile.TemporaryDirectory() as d:
pyarrow.dataset.write_dataset(table, d, format="feather")
print(list(Path(d).iterdir()))
```
Output
```
> python repro.py
PyArrow version: 9.0.0
[]
```
**Reporter**: [David Li](https://issues.apache.org/jira/browse/ARROW-18060) / @lidavidm
#### Related issues:
- [[R] arrow::write_dataset() does nothing with 0 row dataframes in R](https://github.com/apache/arrow/issues/31933) (duplicates)
**Note**: *This issue was originally created as [ARROW-18060](https://issues.apache.org/jira/browse/ARROW-18060). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start with the Python reproduction using pyarrow.dataset.write_dataset and inspect the dataset-writing entry point involved in the empty-table case. Determine the expected behavior for zero rows, including partitioned datasets, and consider the issue complete when that behavior is implemented and verified against the reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100