apache / apache/paimon

[Bug] SparkCatalog converts catalog option keys to lower case

Open
#6,535 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

1.2.0

### Compute Engine

Spark

### Minimal reproduce step

`SparkCatalog::initialize` accepts a `CaseInsensitiveStringMap` as the input and `Options.fromMap(options)` will create an `Options` with lower case keys.
https://github.com/apache/paimon/blob/release-1.2.0/paimon-spark/paimon-spark-common/src/main/java/org/apache/paimon/spark/SparkCatalog.java#L125

### What doesn't meet your expectations?

Sometimes the catalog options should be case sensitive. For example, the `jdbc.socketFactory` option will be passed to the PostgreSQL JDBC driver and the driver only recognizes `socketFactory`, not `socketfactory`.

We can fix this problem by using the original case sensitive map `Options.fromMap(options.asCaseSensitiveMap())`.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Open paimon-spark/paimon-spark-common/src/main/java/org/apache/paimon/spark/SparkCatalog.java and trace SparkCatalog::initialize, focusing on the options conversion described at line 125. Done means preserving option-key casing for case-sensitive settings such as jdbc.socketFactory, then validating the behavior with the relevant project tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering
Issue type
Bug
Difficulty
1/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.