[Bug] Locale-sensitive case conversions break identifier matching, enum parsing and request signing
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Bug description
`String.toLowerCase()` and `String.toUpperCase()` follow the JVM default locale. Under a Turkish or Azeri default, `i` uppercases to the dotted `İ` and `I` lowercases to the dotless `ı`, so any token that is case-folded before being matched, parsed or signed stops matching what it is compared against.
This started as an identifier-matching bug. With `case-sensitive = false`, CDC table mapping, computed columns and the Arrow readers all route through `StringUtils.toLowerCaseIfNeed`, and `CdcRecord.fieldNameLowerCase` is the record side of the same join, so a source column `INDEX` stops mapping to schema field `index` and the column silently nulls out.
Auditing the rest of `paimon-api`, `paimon-common` and `paimon-core` turned up the same conversion in places where it throws rather than mismatching quietly:
- `CoreOptions.partitionMarkDoneActions()` does `PartitionMarkDoneAction.valueOf(x.replace('-','_').toUpperCase())`. Both `success-file` and `done-partition` contain an `i`, so under Turkish this throws `IllegalArgumentException: No enum constant ...PartitionMarkDoneAction.SUCCESS_FİLE`.
- `RowKind.fromShortString("+i")` uppercases to `+İ`, matches no case arm, and throws `UnsupportedOperationException`.
- `DistributedLockDialectFactory` does `JdbcProtocol.valueOf(protocol.toUpperCase())` and loses `SQLITE` and `MARIADB`, so JDBC catalog locking cannot resolve its dialect.
- `OperatingSystem` lowercases `os.name` and then looks for `solaris`, which becomes `solarıs`, so the OS is reported as `UNKNOWN`.
- The DLF request signers lowercase header names to build the signature, option key lookups and format identifiers lowercase user-supplied keys, and system table lookup lowercases the table name. All of these compare the result against a fixed ASCII string.
### Expected behavior
None of these conversions should depend on the operator's locale: they are all machine-facing, so they should pin `Locale.ROOT`.
`BinaryString.toLowerCase` / `toUpperCase` are already safe: the ASCII path uses `Character.toLowerCase` and the non-ASCII fallback pins `Locale.ROOT`, so the SQL `upper()` / `lower()` transforms over user data are not affected.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by auditing locale-sensitive conversions in paimon-api, paimon-common, and paimon-core, focusing on StringUtils.toLowerCaseIfNeed, CdcRecord.fieldNameLowerCase, CoreOptions.partitionMarkDoneActions(), RowKind.fromShortString(), DistributedLockDialectFactory, OperatingSystem, and the DLF request signers. Verify each machine-facing comparison uses Locale.ROOT while leaving BinaryString conversions unchanged. Run the relevant existing tests and confirm identifier matching, enum parsing, dialect resolution, OS detection, and request signing work under Turkish or Azeri defaults.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100