apache / apache/arrow-cookbook

[Python] Recipe showing how to use the filename callback to generate new files every time write_dataset is called

Open
#74 1 comment 1 reaction 0 assignees View on GitHub
python
Dominant language
C++
Stars
108
Forks
49
Avg merge
2h 14m
Merged PRs (30d)
1

Description

Inspired by: https://stackoverflow.com/questions/69184289/pyarrow-overwrites-dataset-when-using-s3-filesystem/69185178#69185178

Contributor guide

Open the contributing guide

Research direction

Search the cookbook for existing Python dataset recipes and the write_dataset entry point, then read the linked Stack Overflow example for the intended filename-callback behavior. Done means a documented recipe demonstrates that repeated write_dataset calls create new files rather than overwriting the dataset.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.