The reduction arguments that are declared and refused
- Dominant language
- Mojo
- Stars
- 1
- Forks
- 0
- PR merge metrics
- PR metrics pending
Description
#330 exposed twelve reductions and declared the full pandas signature on each of them. Seven parameters are declared and raise `NotImplementedError` by name rather than being ignored, which is deliberate and is written up in [`26-the-binding-is-behind-the-library.md`](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/26-the-binding-is-behind-the-library.md). This issue is the list of what it takes to stop refusing them.
They are not one piece of work and the sizes are very different, which is why they are listed rather than merged.
**`skipna=False`.** Every reduction in the kernel skips missing rows, because that is what pandas does by default. `skipna=False` says the opposite: if any row is missing, the answer is missing, and the reduction stops being a fold over the present values and becomes a fold plus a check. It is cheap, since the null count is already known before the loop runs, but it has to be threaded through `AggKind` or handled above it and that choice should be made once for all twelve.
**`min_count`.** `s.sum(min_count=2)` answers NaN when fewer than two rows were present. Same shape as `skipna=False`, and it only applies to `sum` and `prod`.
**`numeric_only=True`.** Only meaningful on a frame, where it drops the non numeric columns before reducing rather than raising on them. It interacts directly with the type unification rule in #330, since the whole reason a mixed frame refuses is that a string column has nothing in common with a number column, and `numeric_only=True` is pandas' answer to exactly that. Implementing it would turn the refusal in `test_a_mix_with_nothing_in_common_says_so` into a working call under a flag.
**`axis=1` on a frame.** A row wise reduction is a different kernel, not the same one pointed sideways. It reduces across columns of possibly different types, one row at a time, and the honest implementation is a kernel rather than a transpose, since transposing an Arrow frame to reduce it would allocate the whole thing.
**A list of quantiles.** `s.quantile([0.1, 0.9])` answers a series indexed by the quantiles rather than a scalar, and `df.quantile([0.1, 0.9])` answers a frame. Different return shape, same kernel called more than once.
**`interpolation` other than `"linear"`.** pandas takes `"lower"`, `"higher"`, `"midpoint"` and `"nearest"`. All four are a different rule for picking between the two bracketing values and none of them needs new machinery, just the rule in the quantile kernel.
**`nunique(dropna=False)`.** Counts the missing value as one distinct value. The distinct count already walks the column, so this is a decision in the walk rather than a new one.
Each of these currently has a test asserting that it still refuses, in `python/tests/test_reductions.py`. Those tests should flip to asserting the answer as each lands, rather than being deleted, so the parameter cannot go back to being silently accepted.
Follow-up to #330. Part of the M6 work tracked in #8.
Contributor guide
Research direction
Start with the refusal tests in python/tests/test_reductions.py and the behavior described in docs/specs/26-the-binding-is-behind-the-library.md. Choose one listed parameter rather than treating this as a single change, then trace the relevant reduction kernel and AggKind discussion. Done means the parameter returns the pandas-compatible answer and its refusal test asserts the result instead.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100