[Bug] External Iceberg columns with mixed-case names are unreadable: "schema mapping is missing projected column"
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Version
master (reproduced on the ~2026-08-16 tip, b9ee837a3e)
### What's Wrong?
After the external-catalog connector refactor (#64304 line), selecting any column whose name contains an uppercase letter from an external Iceberg table fails:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = [INTERNAL_ERROR]schema mapping is missing
projected column 'channelId'; the schema info from FE is inconsistent with the scan projection (file: ...)
```
Plain scalar columns fail the same way as complex ones — any camelCase name is enough. All-lowercase columns read fine.
Root cause, from tracing it in production: `IcebergSchemaUtils.buildCurrentSchema` (fe-connector-iceberg) keys the current-schema (`-1`) dictionary's top-level TField names off the **lowercased** requested names (`IcebergScanPlanProvider.requestedLowerNames` — the column handles are already lowercased). But the BE scan slot names preserve the Iceberg column case, and `ParquetReader::_do_init_reader` (the "schema mapping is missing projected column" guard) compares slot names against the dictionary **verbatim**. So the dict says `channelid`, the slot says `channelId`, and the guard refuses. The doc comment on `requestedLowerNames` asserts the dict names equal the BE scan-slot names "by construction" — that only holds for tables whose columns are already lowercase. The pre-refactor fe-core path built this dictionary from the Doris `Column` names (real case, `ExternalUtil`), which is why the same tables read fine before the decoupling.
### What You Expected?
Column reads work regardless of name case, as they did via the legacy fe-core Iceberg path.
### How to Reproduce?
1. Any external Iceberg table (REST catalog in my case) with a column like `channelId STRING`.
2. `SELECT channelId FROM cat.db.t LIMIT 1;` → the error above. A lowercase sibling column on the same table reads fine.
### Anything Else?
Hit this while rebasing our production build onto master — MongoDB-derived lakehouse tables are almost entirely camelCase, so every one of them was unreadable. We're running a fix in production: emit the schema's own field case for the dictionary's top-level names and keep the lowered name only as the `caseInsensitiveFindField` lookup key (the equality-delete carrier can stay lowercased — it's consumed strictly by field id).
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start with fe-connector-iceberg's IcebergSchemaUtils.buildCurrentSchema and IcebergScanPlanProvider.requestedLowerNames, then inspect ParquetReader::_do_init_reader and its schema-mapping guard. Reproduce the failure with an external Iceberg table containing a camelCase column and compare dictionary names with scan-slot names. Done means mixed-case and lowercase columns both read successfully, including the reported SELECT.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, java, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100