tamnd / tamnd/firepanda

E. Rolling, expanding and exponentially weighted windows

Open
#161 0 comments 0 reactions 0 assignees View on GitHub
area/frame parity
Dominant language
Mojo
Stars
1
Forks
0
PR merge metrics
PR metrics pending

Description

53 callables across three types: `Rolling` 22, `Expanding` 22, `ExponentialMovingWindow` 9. This is the workstream with the most parameter interaction per callable and the one most likely to be underestimated, which is why it is worth stating the arithmetic. The parity checklist has one line reading `rolling with window, min_periods, center, closed, step`. That is one checkbox over five parameters whose interactions are the entire difficulty of the feature, and `rolling` in pandas 3.0 actually takes eight: `window`, `min_periods`, `center`, `win_type`, `on`, `closed`, `step` and `method`.

### What it covers

`Rolling`: `agg`, `aggregate`, `apply`, `corr`, `count`, `cov`, `first`, `kurt`, `last`, `max`, `mean`, `median`, `min`, `nunique`, `pipe`, `quantile`, `rank`, `sem`, `skew`, `std`, `sum`, `var`. `Expanding` is the same list. `ExponentialMovingWindow`: `agg`, `aggregate`, `corr`, `cov`, `mean`, `online`, `std`, `sum`, `var`.

Time based windows, meaning a window given as a frequency string, are M7 and not this issue. The boundary for M6 is everything that does not need a calendar.

### Done when

- [ ] Integer windows with `min_periods`, `center`, `closed` and `step`, and every combination of them that interacts, which is what L3 means here
- [ ] The numerically careful reductions, since a naive online variance over a rolling window loses precision in a way that shows up as a comparison failure rather than as a crash
- [ ] `corr` and `cov` between two series, pairwise handling included
- [ ] `apply` with a user function, `raw` both ways
- [ ] `win_type` weighted windows are allowed to trail and are registered as unimplemented with a milestone if they do
- [ ] L3 at 90 percent, and parameter coverage at 80 percent or better, because an L3 number over a parameter space nobody exercised is not a number

### How this is measured

The target above is an L3 rate over this workstream's callables, reported by `pixi run conformance` in [firepanda-compat](https://github.com/tamnd/firepanda-compat) and enforced by the CI ratchet rather than by a person ticking a box. L3 means every parameter takes every one of its values and the combinations that interact are enumerated. The levels are defined in [01-what-100-percent-means.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/01-what-100-percent-means.md), the counts come from [02-the-surface.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/02-the-surface.md), and the ordering argument for this milestone is in [08-m6.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/08-m6.md).

Anything we deliberately will not do goes in the divergence registry with a reason before this issue closes, and a registered divergence still runs and still has to diverge.

Part of #8, milestone M6.

Contributor guide

Open the contributing guide

Research direction

Start with `pixi run conformance` in firepanda-compat and read `docs/specs/01-what-100-percent-means.md`, `02-the-surface.md`, and `08-m6.md` to understand the measured scope. Work through the listed Rolling, Expanding, and ExponentialMovingWindow callables and parameter combinations. Done means the stated L3 and parameter-coverage targets are met, with deliberate divergences registered before closure.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.