tamnd / tamnd/firepanda

The six transformation arguments that need something the core does not have

Open
#339 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Mojo
Stars
1
Forks
0
Avg merge
1h 31m
Merged PRs (30d)
640

Description

Six pandas arguments on the transformation family refuse by name today, and unlike `axis=1` and `skipna=False` these are not one piece of work. They are grouped here so the list exists in one place rather than being rediscovered.

**`ffill(limit_area=)` and `bfill(limit_area=)`.** `"inside"` fills only gaps that have a real value on both sides, and `"outside"` fills only the leading and trailing ones. Both are a second pass over the mask before the fill runs, and both are cheap once somebody decides where the mask lives.

**`shift(freq=)`.** Shifts the index by a frequency rather than shifting the values by a row count, so it needs the frequency string work in [`15-the-frequency-string.md`](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/15-the-frequency-string.md) and a datetime index. It is a different operation wearing the same name.

**`shift(periods=[...], suffix=)`.** A list of periods gives a frame with one column per period, named with the suffix. It changes the shape of the answer, so it is not a parameter on the existing call so much as a second call that loops.

**`shift(fill_value=)`.** Puts a value in the gap rather than leaving it missing, and the core already has this: `firepanda/kernel/shift.mojo` takes a fill. What it needs is a way to carry an arbitrary Python value across the boundary and turn it into a scalar of the column's type, which is the same problem `fillna` has and should be solved once for both.

**`dropna(how="all")` and `dropna(thresh=)`.** `DataFrame.drop_nulls` in the core implements only `how="any"`. `how="all"` keeps a row unless every column in it is missing, and `thresh=n` keeps a row with at least n present values, which is the general form both are special cases of. This is a core change in `firepanda/frame/frame.mojo` rather than a binding one.

Falls out of [#336](https://github.com/tamnd/firepanda/pull/336) and part of [#334](https://github.com/tamnd/firepanda/issues/334).

Contributor guide

Open the contributing guide

Research direction

Start by separating the six argument groups and read docs/specs/15-the-frequency-string.md for the frequency dependency. Inspect firepanda/kernel/shift.mojo for fill handling and firepanda/frame/frame.mojo for drop_nulls; the work is done when the listed pandas arguments have their stated semantics across the transformation family.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.