tamnd / tamnd/firepanda

A1. The Index type, 73 callables

Open
#154 1 comment 0 reactions 0 assignees View on GitHub
area/frame parity
Dominant language
Mojo
Stars
1
Forks
0
PR merge metrics
PR metrics pending

Description

`Index` is 91 public names and 73 callables, and document 06 in the specification does not mention 30 of them. That is not an oversight in the document so much as a consequence of treating the index as a property of a frame rather than as an object with an API. `loc` is built on `get_loc` and `get_indexer`, `reindex` is built on `get_indexer`, a merge on an index is built on `get_indexer_non_unique`, and `unstack` is built on `get_level_values`. The index API is not an add on to the index work, it is the index work.

This lands third in M6, after the error types and the cheap surface, because everything else in the milestone leans on it. `groupby(...).agg(...)` produces an indexed frame, `value_counts` produces an indexed series, `describe` produces an indexed frame. Building those against a frame type with no index means building them twice.

### The 30 names document 06 never mentions

`union`, `intersection`, `difference`, `symmetric_difference`, `get_loc`, `get_indexer`, `get_indexer_non_unique`, `get_level_values`, `slice_locs`, `slice_indexer`, `asof_locs`, `append`, `delete`, `putmask`, `has_duplicates`, `identical`, `is_`, `inferred_type`, `nlevels`, `names`, `set_names`, `sortlevel`, `to_flat_index`, `to_series`, `ravel`, `view`, `nbytes`, `drop`, `get_slice_bound`, `get_indexer_for`.

### Done when

- [x] The four set operations, with the sort and name rules pandas uses, which are fiddly and are where the first bugs will be
- [x] `get_loc` and the three `get_indexer` forms, since these are the primitives everything else calls
- [x] Monotonic and unique fast paths, because an index that is sorted and unique is the common case and the reason an index is worth having
- [x] Index equality and `identical`, which differ, and the difference matters to `align`
- [ ] L3 at 90 percent or better for the workstream, with the remaining 10 percent named rather than unexplained

### How this is measured

The target above is an L3 rate over this workstream's callables, reported by `pixi run conformance` in [firepanda-compat](https://github.com/tamnd/firepanda-compat) and enforced by the CI ratchet rather than by a person ticking a box. L3 means every parameter takes every one of its values and the combinations that interact are enumerated. The levels are defined in [01-what-100-percent-means.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/01-what-100-percent-means.md), the counts come from [02-the-surface.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/02-the-surface.md), and the ordering argument for this milestone is in [08-m6.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/08-m6.md).

Anything we deliberately will not do goes in the divergence registry with a reason before this issue closes, and a registered divergence still runs and still has to diverge.

Part of #8, milestone M6.

Contributor guide

Open the contributing guide

Research direction

Start by reading specs/01-what-100-percent-means.md, specs/02-the-surface.md, and specs/08-m6.md to understand the callable counts, conformance levels, and milestone ordering. Run `pixi run conformance` in firepanda-compat and use the listed Index callables as the workstream scope. Done means L3 reaches at least 90%, with remaining gaps named and deliberate divergences recorded in the divergence registry.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.