B1. The 49 string methods that need no regex
- Dominant language
- Mojo
- Stars
- 1
- Forks
- 0
- PR merge metrics
- PR metrics pending
Description
The `.str` accessor is 57 methods and 49 of them do not touch a regular expression. Those 49 are not blocked on the regex engine and should not wait for it, which is why this is split from the RE2 issue and why the two run in parallel.
The specification says the string namespace ships whole and it is right, because half of `.str` is worse than none of it: a user discovers the gap after they have already ported their program. Whole means these 49 plus the eight in the RE2 issue, both landed before M6 closes.
### The 49
`capitalize`, `casefold`, `cat`, `center`, `decode`, `encode`, `endswith`, `find`, `get`, `get_dummies`, `index`, `isalnum`, `isalpha`, `isascii`, `isdecimal`, `isdigit`, `islower`, `isnumeric`, `isspace`, `istitle`, `isupper`, `join`, `len`, `ljust`, `lower`, `lstrip`, `normalize`, `pad`, `partition`, `removeprefix`, `removesuffix`, `repeat`, `rfind`, `rindex`, `rjust`, `rpartition`, `rsplit`, `rstrip`, `slice`, `slice_replace`, `startswith`, `strip`, `swapcase`, `title`, `translate`, `upper`, `wrap`, `zfill`, and the non regex path of `split`.
### The parts that are actually hard
Case conversion is not ASCII. `casefold` on a German sharp s produces two characters, `upper` on it produces two characters, and a length preserving implementation is wrong. Same for the Turkish dotless i under any locale aware reading, which pandas does not do and we should not either, so the divergence is that we match Python's locale independent behaviour exactly.
`normalize` is the four Unicode normalization forms and it needs the Unicode data tables, so it is the one method here with a data dependency.
The `is*` predicates follow Python's `str` definitions rather than anything simpler, and `isdecimal`, `isdigit` and `isnumeric` differ from each other on characters most people have never seen. The corpus has a Unicode frame precisely for this.
### Done when
- [ ] All 49 present and matching pandas on the ASCII, Unicode, null heavy and pattern frames in the corpus
- [ ] Null propagation matches, since a null in equals a null out for every one of these and it is the rule most likely to be got wrong once
- [ ] Case conversion is correct for the multi character expansions rather than assuming a one to one mapping
- [ ] L3 at 100 percent
### How this is measured
The target above is an L3 rate over this workstream's callables, reported by `pixi run conformance` in [firepanda-compat](https://github.com/tamnd/firepanda-compat) and enforced by the CI ratchet rather than by a person ticking a box. L3 means every parameter takes every one of its values and the combinations that interact are enumerated. The levels are defined in [01-what-100-percent-means.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/01-what-100-percent-means.md), the counts come from [02-the-surface.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/02-the-surface.md), and the ordering argument for this milestone is in [08-m6.md](https://github.com/tamnd/firepanda-compat/blob/main/docs/specs/08-m6.md).
Anything we deliberately will not do goes in the divergence registry with a reason before this issue closes, and a registered divergence still runs and still has to diverge.
Part of #8, milestone M6.
Contributor guide
Research direction
Start by locating the `.str` accessor and the existing corpus/conformance entry points, then run `pixi run conformance` to establish the current L3 baseline. Use the Unicode, null-heavy, ASCII, and pattern frames to guide coverage; done means all listed methods match pandas, null propagation and multi-character case expansions are correct, and L3 reaches 100 percent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100