tamnd / tamnd/firepanda

A2. MultiIndex, 86 callables

Open
#155 0 comments 0 reactions 0 assignees View on GitHub
area/frame parity
Dominant language
Mojo
Stars
1
Forks
0
PR merge metrics
PR metrics pending

Description

86 callables, and the largest single type in the pandas surface after `DataFrame` and `Series`. It is separate from the `Index` issue because it can start once `Index` has its primitives and because it is the piece most likely to be underestimated: a `MultiIndex` is not an index of tuples, it is codes and levels, and every operation on it is an operation on the codes.

The target here is 70 percent rather than 100, and that is deliberate. Full `MultiIndex` conformance includes lexsort depth, partial indexing with slices, `IndexSlice`, and the interaction between all three, and that is a milestone of its own. What M6 owes is the part real programs use: the result of a groupby on two keys, the result of an unstack, and selecting from both.

### Done when

- [ ] `from_arrays`, `from_tuples`, `from_product`, `from_frame`, since this is how one gets built
- [ ] Codes and levels as the storage, with `remove_unused_levels` and `set_levels`
- [ ] `get_level_values`, `droplevel`, `swaplevel`, `reorder_levels`, `sortlevel`
- [ ] Partial selection on the first level, and `xs` at any level
- [ ] `is_lexsorted` and the sortedness state, because the fast paths depend on it and pandas raises `UnsortedIndexError` when it is missing
- [ ] L3 at 70 percent or better, with everything below it registered as unimplemented with a milestone rather than left silent

### How this is measured

The target above is an L3 rate over this workstream's callables, reported by `pixi run conformance` in [firepanda-compat](https://github.com/tamnd/firepanda-compat) and enforced by the CI ratchet rather than by a person ticking a box. L3 means every parameter takes every one of its values and the combinations that interact are enumerated. The levels are defined in [01-what-100-percent-means.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/01-what-100-percent-means.md), the counts come from [02-the-surface.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/02-the-surface.md), and the ordering argument for this milestone is in [08-m6.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/08-m6.md).

Anything we deliberately will not do goes in the divergence registry with a reason before this issue closes, and a registered divergence still runs and still has to diverge.

Part of #8, milestone M6.

Contributor guide

Open the contributing guide

Research direction

Start with the MultiIndex callable checklist and read the linked specs 01-what-100-percent-means.md, 02-the-surface.md, and 08-m6.md to understand the L3 target and milestone boundaries. Run `pixi run conformance` in firepanda-compat to establish the baseline, then cover construction, codes and levels, selection, and sorting. Done means at least 70% L3 coverage, registered unimplemented or divergent behavior, and a passing CI ratchet.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas
Domain
data, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.