tamnd / tamnd/firepanda

The transformation family, which is the same door problem as the reductions

Open
#334 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Mojo
Stars
1
Forks
0
PR merge metrics
PR metrics pending

Description

#330 found that `AggKind` had seventeen reductions implemented and the Python extension exposed none of them, so `s.sum()` did not work against a library that computes it correctly. The same reading of the source says the transformations are in the same state, and this is that list.

Each of these exists in the core under its own name and is not reachable from a pandas program under the pandas name.

**On `Series`:**

| core | pandas |
|---|---|
| `drop_nulls` | `dropna` |
| `fill_null` | `fillna` |
| `fill_forward` | `ffill` |
| `fill_backward` | `bfill` |
| `cast` | `astype` |
| `is_null` | `isna` |
| `is_not_null` | `notna` |
| `shift` | `shift` |
| `diff` | `diff` |
| `pct_change` | `pct_change` |
| `cumsum` `cumprod` `cummax` `cummin` | same |
| `is_monotonic_increasing` `is_monotonic_decreasing` | same |
| `argsort` `sort_values` `take` `rename` | same |

**On `DataFrame`:** `dropna`, `fillna`, `astype`, `sort_values`, `take`, `rename`, `drop`, `describe`, `join`, `groupby`.

Most of these are a rename at the boundary rather than a translation, since the pandas name and the core name mean the same operation. The ones that are not:

- `fillna` takes a value, a method, a limit and a dict of per column values in pandas, and `fill_null` takes a value. The extra shapes should refuse by name the way the reduction parameters in #332 do rather than being silently dropped.
- `astype` already went through the missing value policy work in #300 and #326, so the pandas facing spelling has to keep the widening rule those landed rather than calling the core cast directly.
- `describe` answers a frame of eight statistics with a specific index and specific dtype rules, so it is a real piece of work rather than a rename, and it is worth doing because it is the first thing many people type.
- `groupby` on a frame answers a grouped object rather than a frame, which needs the indexed group by shape.

**Separately, a `.dt` accessor.** `Series` has fifteen `dt_*` methods already implemented. `dt` is 42 names in the pandas inventory. An accessor object that forwards `s.dt.year` to `s.dt_year()` makes fifteen of those resolve immediately, and it is a small object rather than any new computation.

The reason to do these as one slice rather than fifteen is the same reason the reductions were one slice: they share a boundary shape, and building fifteen separate doors invites fifteen chances for them to drift.

The measurement to expect is in the conformance board's `resolution` and `signature` sections, where these names currently report absent.

Follow-up to #330, and the next slice argued for in [`26-the-binding-is-behind-the-library.md`](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/26-the-binding-is-behind-the-library.md). Part of the M6 work tracked in #8.

Contributor guide

Open the contributing guide

Research direction

Start with the transformation names in the issue and the conformance board’s resolution and signature sections, then read docs/specs/26-the-binding-is-behind-the-library.md and the boundary code exposing the Python pandas API. Compare the existing core methods with the listed Series and DataFrame names, including the dt accessor. Done means the listed names resolve with the stated parameter, dtype, grouped-object, and accessor behavior rather than reporting absent.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
api, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.