expression: reduce UTF8 string builtin allocations by removing []rune conversion
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Enhancement
## Enhancement
UTF8 string builtins in `pkg/expression` still use `[]rune(str)` in hot paths (both row-based and vectorized execution) for rune counting and slicing.
This causes avoidable allocations and extra CPU/GC overhead.
### Proposed improvement
Replace `[]rune`-based logic with zero-allocation UTF8 operations:
- `len([]rune(str))` -> `utf8.RuneCountInString(str)`
- `string([]rune(str)[:n])` / `string([]rune(str)[n:])` -> byte slicing at rune boundaries via a helper like `runeByteIndex(str, n)`
- Apply consistently in scalar + vectorized paths for:
- `LEFT`, `RIGHT`
- `LOCATE`
- `SUBSTR` / `MID` / `SUBSTRING`
- `INSERT`
- `LPAD`, `RPAD`
- `CHAR_LENGTH`
### Expected result
- Fewer allocations in expression evaluation
- Lower GC pressure
- No SQL behavior change for UTF8 semantics
### Validation plan
- Run targeted unit tests for affected UTF8 string builtins (scalar and vectorized)
- Add/compare benchmarks (allocs/op, ns/op) before and after
- Verify edge cases: multibyte characters, negative/out-of-range positions, empty substring/pad string, collation-sensitive locate behavior
Contributor guide
Research direction
Start in pkg/expression by locating the []rune conversions used by the listed UTF8 string builtins, and compare their scalar and vectorized paths. Run targeted unit tests and inspect existing benchmarks before checking multibyte, boundary, padding, and collation-sensitive cases. Done means the affected paths avoid the stated allocations while preserving UTF8 SQL behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100