apache / apache/arrow

[Python][Documentation] Clarify language around use_deprecated_int96_timestamps

Open
#35,645 2 comments 0 reactions 0 assignees View on GitHub
Component: Documentation Component: Python Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Describe the enhancement requested

PyArrow's ParquetWriter has an argument, `use_deprecated_int96_timestamps`, which is used to write timestamps as int96 which is deprecated in Parquet. I think the docstring may be confusing to new users as it's not entirely clear if this is something deprecated in PyArrow that may be removed at some point or if it's just deprecated elsewhere and PyArrow maintains support for it. The case is the latter.

See the docstring:

https://github.com/apache/arrow/blob/f59e37f8a676a13f0bd82a09c211ddb45c6b48ac/python/pyarrow/parquet/core.py#L772-L774

I think the Parquet section in the PyArrow docs does a better job of explaining this feature,

> Older Parquet implementations use ``INT96`` based storage of
> timestamps, but this is now deprecated. This includes some older
> versions of Apache Impala and Apache Spark. To write timestamps in
> this format, set the ``use_deprecated_int96_timestamps`` option to
> ``True`` in ``write_table``.

I think the docstring could be improved by using similar language to the above.

### Component(s)

Documentation, Python

Contributor guide

Open the contributing guide

Research direction

Open python/pyarrow/parquet/core.py at the ParquetWriter docstring for use_deprecated_int96_timestamps and compare its wording with the Parquet documentation section quoted in the issue. Update the docstring to clarify that INT96 is deprecated in Parquet and retained for compatibility with older implementations, then verify the generated documentation reflects the clarified language.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.