oceanbase / oceanbase/seekdb

Restore support for commonly used UTF-8 collations

Open
#1,271 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dima type: task
Dominant language
C++
Stars
2.9k
Forks
339
Avg merge
1d 19h
Merged PRs (30d)
52

Description

Background

Parent work item: [Remove some charsets](.

To reduce binary size, the parent work item streamlined charset/collation support. Currently, the available collations for utf8mb4 are too few. It is necessary to evaluate and restore the UTF-8 collations commonly used in business scenarios.

Research Conclusion

  • utf8mb4_unicode_ci: Common in MySQL 5.x/compatible migration scenarios. Should be prioritized for support.
  • utf8mb4_0900_ai_ci: The default collation for MySQL 8.0. Should be prioritized for support in MySQL 8.0 compatible migration scenarios.
  • utf8mb4_unicode_520_ci: Used in some legacy systems. Decision on support should be based on compatibility benefits versus binary size increase.
  • Further evaluation is possible for utf8mb4_0900_as_cs and utf8mb4_0900_bin. Do not restore language-specific collations in bulk for now.

Implementation Requirements

  • Implement the real comparison semantics according to the corresponding Unicode Collation Algorithm version. Must not be simple aliases to utf8mb4_general_ci.
  • Correctly handle case sensitivity, accents, and PAD SPACE / NO PAD semantics.
  • Ensure compatibility for SHOW COLLATION, DDL, schema dump/restore, and client metadata.

Acceptance Criteria

  • Add comprehensive tests covering comparison, sorting, GROUP BY, DISTINCT, unique keys, etc.
  • Add compatibility tests for table creation, table alteration, and schema dump/restore.
  • Provide the binary size increment introduced by each collation to support future trade-off decisions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source files, tests, or entry points are named. Start by locating the existing utf8mb4 collation implementation and compatibility handling, then review how collations are exposed and tested; done means real Unicode comparison behavior, DDL and metadata compatibility, comprehensive comparison and schema tests, and reported binary-size impact for each collation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, mysql, sql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.