tamnd / tamnd/firepanda

K. api.types predicates and the option system

Open
#165 0 comments 0 reactions 0 assignees View on GitHub
area/dtype parity
Dominant language
Mojo
Stars
1
Forks
0
PR merge metrics
PR metrics pending

Description

45 predicates in `pandas.api.types` and 70 option keys under `describe_option`. Neither is glamorous and both are load bearing for a reason that has nothing to do with our users writing them: third party libraries call them. seaborn, scikit-learn, statsmodels and every plotting wrapper in the ecosystem branch on `is_numeric_dtype` and `is_categorical_dtype` before they touch the data. A frame that fails those checks is a frame those libraries reject without a useful error.

This is last in M6 because handing a frame to a third party library needs the Python front door from M3, and it is in M6 rather than later because it is cheap and because leaving it out makes every one of those call paths a surprise.

### What it covers

The 45 names include `infer_dtype`, `pandas_dtype`, `union_categoricals`, the four dtype classes, and the `is_*` family in full. Some of them are deprecated in pandas 3.0, `is_sparse` and `is_categorical_dtype` among them, and they are implemented anyway with the same deprecation warning, because code in the wild calls them and a warning is the compatible behaviour.

The 70 options are not all worth supporting. The eight load bearing ones are the display options that affect `repr`, `mode.copy_on_write`, `future.infer_string`, and the compute options. The rest exist, accept a value, and are honest about being inert, which is better than an `OptionError` from a library that sets an option defensively at import time.

### Done when

- [ ] All 45 predicates present and returning what pandas returns for every dtype in the corpus, including on non frame inputs like lists and scalars, since that is how they get called
- [ ] Deprecated predicates work and warn
- [ ] `get_option`, `set_option`, `reset_option`, `describe_option` and `option_context` over the full key list
- [ ] The eight load bearing options actually change behaviour, and every other key round trips without being a lie about doing something
- [ ] A smoke test that hands a firepanda frame to a library that uses these, once M3 makes that possible

### How this is measured

The target above is an L3 rate over this workstream's callables, reported by `pixi run conformance` in [firepanda-compat](https://github.com/tamnd/firepanda-compat) and enforced by the CI ratchet rather than by a person ticking a box. L3 means every parameter takes every one of its values and the combinations that interact are enumerated. The levels are defined in [01-what-100-percent-means.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/01-what-100-percent-means.md), the counts come from [02-the-surface.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/02-the-surface.md), and the ordering argument for this milestone is in [08-m6.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/08-m6.md).

Anything we deliberately will not do goes in the divergence registry with a reason before this issue closes, and a registered divergence still runs and still has to diverge.

Part of #8, milestone M6.

Contributor guide

Open the contributing guide

Research direction

Start with the pandas.api.types predicate surface and the get_option, set_option, reset_option, describe_option, and option_context entry points named in the issue. Run pixi run conformance in firepanda-compat and consult the linked surface and M6 specifications; done means the 45 predicates, 70 keys, warnings, load-bearing behavior, round trips, and smoke test meet the stated checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
api, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.