The six transformation arguments that need something the core does not have
- Dominant language
- Mojo
- Stars
- 1
- Forks
- 0
- Avg merge
- 1h 31m
- Merged PRs (30d)
- 640
Description
Six pandas arguments on the transformation family refuse by name today, and unlike `axis=1` and `skipna=False` these are not one piece of work. They are grouped here so the list exists in one place rather than being rediscovered.
**`ffill(limit_area=)` and `bfill(limit_area=)`.** `"inside"` fills only gaps that have a real value on both sides, and `"outside"` fills only the leading and trailing ones. Both are a second pass over the mask before the fill runs, and both are cheap once somebody decides where the mask lives.
**`shift(freq=)`.** Shifts the index by a frequency rather than shifting the values by a row count, so it needs the frequency string work in [`15-the-frequency-string.md`](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/15-the-frequency-string.md) and a datetime index. It is a different operation wearing the same name.
**`shift(periods=[...], suffix=)`.** A list of periods gives a frame with one column per period, named with the suffix. It changes the shape of the answer, so it is not a parameter on the existing call so much as a second call that loops.
**`shift(fill_value=)`.** Puts a value in the gap rather than leaving it missing, and the core already has this: `firepanda/kernel/shift.mojo` takes a fill. What it needs is a way to carry an arbitrary Python value across the boundary and turn it into a scalar of the column's type, which is the same problem `fillna` has and should be solved once for both.
**`dropna(how="all")` and `dropna(thresh=)`.** `DataFrame.drop_nulls` in the core implements only `how="any"`. `how="all"` keeps a row unless every column in it is missing, and `thresh=n` keeps a row with at least n present values, which is the general form both are special cases of. This is a core change in `firepanda/frame/frame.mojo` rather than a binding one.
Falls out of [#336](https://github.com/tamnd/firepanda/pull/336) and part of [#334](https://github.com/tamnd/firepanda/issues/334).
Contributor guide
Research direction
Start by separating the six argument groups and read docs/specs/15-the-frequency-string.md for the frequency dependency. Inspect firepanda/kernel/shift.mojo for fill handling and firepanda/frame/frame.mojo for drop_nulls; the work is done when the listed pandas arguments have their stated semantics across the transformation family.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100