apache / apache/doris

[Feature] Support user-specified characterEncoding in JDBC Catalog URL

Open Beginner friendly
#66,993 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

## Is your feature request related to a problem?

Doris currently hardcodes `characterEncoding=utf-8` for MySQL/OceanBase JDBC catalogs. Users connecting to legacy MySQL tables stored in latin1 (or gbk, etc.) cannot read their data correctly, because Doris always forces UTF-8 and even produces a duplicated parameter when the user specifies their own value:

```
jdbc:mysql://host:port/db?characterEncoding=latin1
```

becomes

```
jdbc:mysql://host:port/db?characterEncoding=latin1&characterEncoding=utf-8
```

## Describe the solution you'd like

Respect the `characterEncoding` value explicitly provided by the user in `jdbc_url`. Concretely, in `fe/fe-core/src/main/java/org/apache/doris/catalog/JdbcResource.java`, the injection logic:

```java
newJdbcUrl = checkAndSetJdbcParam(dbType, newJdbcUrl, "characterEncoding", "utf-8");
```

should skip injection when `characterEncoding` is already present (with any value), and only append `characterEncoding=utf-8` as the default when it is absent.

## Describe alternatives you've considered

1. Keep forcing UTF-8 but at least replace (not append) any user-specified value — however this would still not serve latin1 users.
2. Add a new catalog property (e.g. `jdbc_character_encoding`) to control it — heavier, and redundant since `jdbc_url` already carries the parameter.

## Additional context

- Root cause: `checkAndSetJdbcParam` checks `jdbcUrl.contains("characterEncoding=utf-8")` literally, so any other value (latin1/gbk/...) triggers appending a duplicate.
- Legacy latin1 tables are common in older systems; this limitation blocks migration to Doris's JDBC federated queries.
- Current behavior exists on both branch-2.x and master.

Contributor guide

Open the contributing guide

Research direction

Start in fe/fe-core/src/main/java/org/apache/doris/catalog/JdbcResource.java and inspect checkAndSetJdbcParam, especially the characterEncoding call. Verify the JDBC URL behavior when characterEncoding is already present with a non-UTF-8 value and when it is absent. Done means user-specified values are preserved without duplication and the UTF-8 default is added only when missing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
databases
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.