apache / apache/arrow-cookbook

[Python] test failure in io.rst

Open
#140 1 comment 0 reactions 0 assignees View on GitHub
python
Dominant language
C++
Stars
108
Forks
49
Avg merge
2h 14m
Merged PRs (30d)
1

Description

`make pytest` gives me the following error:
```
**********************************************************************
File "io.rst", line 511, in default
Failed example:
import pyarrow as pa
import pyarrow.dataset as ds

ds.write_dataset(data, "./partitioned", format="parquet",
partitioning=ds.partitioning(pa.schema([("year", pa.int16())])))
Exception raised:
Traceback (most recent call last):
File "/home/antoine/miniconda3/envs/cookbook/lib/python3.9/doctest.py", line 1336, in __run
exec(compile(example.source, filename, "single",
File "", line 4, in
ds.write_dataset(data, "./partitioned", format="parquet",
File "/home/antoine/miniconda3/envs/cookbook/lib/python3.9/site-packages/pyarrow/dataset.py", line 878, in write_dataset
_filesystemdataset_write(
File "pyarrow/_dataset.pyx", line 3408, in pyarrow._dataset._filesystemdataset_write
File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Could not write to ./partitioned as the directory is not empty and existing_data_behavior is to error
**********************************************************************
```

Contributor guide

Open the contributing guide

Research direction

Start by running `make pytest` and inspect the example at line 511 of `io.rst`, where the dataset-writing doctest fails because `./partitioned` is not empty. Check how this example is prepared and rerun the doctest after the relevant documentation or test setup is adjusted. Done means the `io.rst` doctest passes under `make pytest`.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.