WordPress / WordPress/sqlite-database-integration
Limit character-set support to UTF-8, ASCII, and binary
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 363
- Forks
- 67
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 7
Description
Problem
The SQLite driver currently accepts MySQL character sets that it cannot represent safely. The SQL lexer expects UTF-8 input, and SQLite text values are stored as UTF-8.
This becomes visible in WordPress when column metadata reports a non-UTF-8 character set. wpdb::strip_invalid_text() then generates CONVERT(... USING charset) queries containing the original encoded bytes, which the driver cannot parse or convert correctly.
Supported character sets
Limit text support to character sets whose encoded values are valid UTF-8:
utf8mb4utf8mb3/utf8asciibinaryvalues and types
Other character sets, including Big5, KOI8-R, Latin-1, CP1251, Hebrew, and TIS-620, are unsupported. They should not be accepted for columns or CONVERT(... USING charset) expressions.
Expected behavior
- Reject unsupported character sets in
CREATE TABLEandALTER TABLEwith a clear driver error. - Reject unsupported character sets in
CONVERT(... USING charset). - Implement correct behavior for supported UTF-8, ASCII, and binary conversions instead of treating every conversion as a no-op.
- Ensure column metadata only exposes supported character sets.
- Define the behavior for existing SQLite databases whose emulated metadata contains unsupported character sets.
- Add coverage for DDL, metadata,
CONVERT, and WordPress CRUD behavior at the supported-character-set boundary.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the SQLite driver's character-set handling in CREATE TABLE, ALTER TABLE, metadata, and CONVERT(... USING charset), then review the WordPress wpdb::strip_invalid_text() path and related CRUD behavior. Done means supported UTF-8, ASCII, and binary cases are covered, unsupported sets are rejected clearly, metadata behavior is defined for existing databases, and tests cover the listed boundaries.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php, sqlite
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100