ClickHouse / ClickHouse/clickhouse-java

Version client-v2-0.9.4 changes order of query settings as URL params in CH HTTP request

未关闭
#2,732 4 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

area:general client-api-v2
主要语言
Java
星标
1.6k
派生
637
平均合并
2 天 23 小时
30 天内合并 PR
29

描述

Once we upgraded from client-v2-0.9.0 to client-v2-0.9.4 we are seeing that in the final HTTP request made to clickhouse the order of the URL params is changing resulting in the change in behavior.

Since clickhouse processes the query settings in the same order, giving preference to the parameter coming later, in our case we are seeing the profile setting parameter going to the end resulting in the change in query result.

Query string in version 0.9.0

http://localhost:18123?query_id=ddcb9fb8-b875-477f-b221-839b6ae2b375_http-nio-4010-exec-10_143&compress=1&role=event_interactive_reader&profile=event_interactive_reader_profile&prefer_global_in_and_join=0&log_comment=; role_s: [event_interactive_reader]; profile_s: event_interactive_reader_profile

Query string in version 0.9.4
http://localhost:18123?query_id=2dd8034a-c44f-4524-a51d-9022cb50b402_http-nio-4010-exec-1_449&compress=1&role=event_interactive_reader&log_comment=; role_s: [event_interactive_reader]; profile_s: event_interactive_reader_profile&prefer_global_in_and_join=1&profile=event_interactive_reader_profile

If you see the profile parameter is going to the end resulting in the settings from the profile not allowing to override the prefer_global_in_and_join=1 value. Its defaulting to the value in the setting.

In version 0.9.0 the order of adding the setting to QuerySettings through serverSetting method was retained, but its breaking in 0.9.4

Steps to reproduce

Have a profile settings and some query setting overriding the value in setting. Set the query setting on QuerySettings at the end in version 0.9.0 this is preserved in the final URL, but its not in 0.9.4.

Error Log or Exception StackTrace

No errors as such, but due to the. different query settings being applied the result from clickhouse would vary. In our case the value of prefer_global_in_and_join was changing resulting in no data being returned.

Expected Behaviour
Code Example

 QuerySettings querySettings = new QuerySettings();
querySettings.setFormat(clickHouseFormat);
querySettings.setDBRoles(Set.of("event_interactive_reader"));

String readSettingsProfile = "event_interactive_reader_profile";
settings.put("profile", "event_interactive_reader_profile");
settings.put("prefer_global_in_and_join", "1");

querySettings.logComment("; role_s: [event_interactive_reader]; profile_s: event_interactive_reader_profile");
querySettings.setQueryId("2dd8034a-c44f-4524-a51d-9022cb50b402_http-nio-4010-exec-1_449");
QueryResponse queryResponse = client.query("select * from database.events", querySettings).get(getClientSocketTimeout(client), TimeUnit.MILLISECONDS);

If on the profile event_interactive_reader_profile if prefer_global_in_and_join is 0 then in client version 0.9.4 it will always be 0 as profile ends up in the end.

Configuration
Client Configuration
com.clickhouse.client.api.Client client = new com.clickhouse.client.api.Client.Builder()
                .addEndpoint(endpoint)
                .setUsername(username)
                .setPassword(password)
                .setMaxConnections(10)
             
                .setConnectionTTL(getPropertyOrDefault("clickhouse.client.connection.ttlMillis", 0L, Long::valueOf), ChronoUnit.MILLIS)
                .setKeepAliveTimeout(getPropertyOrDefault("clickhouse.client.keepAlive.timeoutMillis", -1L, Long::valueOf), ChronoUnit.MILLIS)
                .setConnectionReuseStrategy(ConnectionReuseStrategy.valueOf(getPropertyOrDefault("clickhouse.client.connection.reuseStrategy", "LIFO", String::valueOf)))
                .setConnectTimeout(getPropertyOrDefault("clickhouse.client.connection.timeoutMillis", 5000L, Long::valueOf), ChronoUnit.MILLIS)
                .setSocketTimeout(getPropertyOrDefault("clickhouse.client.default.socketTimeoutMillis", 30000L, Long::valueOf), ChronoUnit.MILLIS)
                .build();
Environment
  • Cloud
  • Client version: client-v2-0.9.4
  • Language version: Java
  • OS: Debian
ClickHouse Server
  • ClickHouse Server version: 25.3
  • ClickHouse Server non-default settings, if any:
  • CREATE TABLE statements for tables involved:
  • Sample data for all these tables, use clickhouse-obfuscator if necessary

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 QuerySettings 开始,尤其是 serverSetting 路径,并跟踪在 client.query 发送请求之前设置是如何组装的。使用 client-v2-0.9.4 重现所提供的 profile 和 prefer_global_in_and_join 示例,并将最终 URL 的顺序与 0.9.0 进行比较。当插入顺序得到保留,使后续设置仍保持其预期优先级时,即视为完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
java
领域
api, databases
Issue 类型
缺陷
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
42/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。