Add `tail` method on DataFrame
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
Is there a better way we could do this? Maybe add something upstream if necessary?
As I'm thinking of it, I don't know that this operation is necessarily well defined. Just like with `limit` when you call it multiple times on a large dataframe you get different results, I would expect different results from multiple calls here.
If we do put this in, I would suggest adding more text to the description to explain why this is an expensive operation - that it performs a collect to determine the size of the dataframe.
_Originally posted by @timsaucer in https://github.com/apache/datafusion-python/pull/915#discussion_r1798327215_
Contributor guide
Research direction
Start with the discussion in datafusion-python PR #915, where this proposal originated. Clarify the semantics of repeated tail calls and the cost of collecting to determine DataFrame size before choosing an implementation. Done means an agreed tail API with documentation explaining its behavior and expense.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100