H. Statistics and the rest of the frame
- Dominant language
- Mojo
- Stars
- 1
- Forks
- 0
- PR merge metrics
- PR metrics pending
Description
61 callables that do not group into a namespace and are therefore easy to leave until last and then discover are half the reason people use pandas. `rank`, `cut`, `qcut`, `factorize`, `interpolate`, `replace`, `describe`, `corr`, `cov`, `duplicated`, `nunique`, `mode`, `quantile`, `clip`, `round`, `pct_change`, `diff`, `shift`, `where`, `mask`, `between`, `isin`, `nlargest`, `nsmallest`, `idxmin`, `idxmax`, `cummax` and the rest of that surface.
### The ones with real difficulty in them
`rank` has five methods and the `ties` behaviour is the whole feature. `cut` and `qcut` have bin edge inclusion rules that are off by one in the obvious implementation and produce a categorical, which ties this to the categoricals issue. `interpolate` has around a dozen methods and the linear one is not the hard part. `replace` accepts scalars, lists, dicts and nested dicts and the dict form is where the behaviour lives. `describe` returns a different set of rows per dtype and is the single most common thing a user types after loading a file, so being close is not enough.
### Done when
- [ ] `rank` with all five methods, `ascending` and `pct`, and `na_option`
- [ ] `cut` and `qcut` with the pandas bin edge rules, returning a proper categorical
- [ ] `describe` matching row for row per dtype, including the mixed dtype frame case
- [ ] `interpolate` with the methods that do not need scipy, and the ones that do registered as unimplemented rather than silently absent
- [x] `replace` in all four argument forms
- [ ] `corr` and `cov` with `method` and `min_periods`
- [ ] L3 at 90 percent
### How this is measured
The target above is an L3 rate over this workstream's callables, reported by `pixi run conformance` in [firepanda-compat](https://github.com/tamnd/firepanda-compat) and enforced by the CI ratchet rather than by a person ticking a box. L3 means every parameter takes every one of its values and the combinations that interact are enumerated. The levels are defined in [01-what-100-percent-means.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/01-what-100-percent-means.md), the counts come from [02-the-surface.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/02-the-surface.md), and the ordering argument for this milestone is in [08-m6.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/08-m6.md).
Anything we deliberately will not do goes in the divergence registry with a reason before this issue closes, and a registered divergence still runs and still has to diverge.
Part of #8, milestone M6.
Contributor guide
Research direction
Start with the listed callables and run `pixi run conformance` in firepanda-compat to establish the current L3 coverage. Read the referenced specs `01-what-100-percent-means.md`, `02-the-surface.md`, and `08-m6.md`; done means the listed behaviors are covered, L3 reaches 90 percent, and any deliberate divergences are registered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100