ClickHouse / ClickHouse/clickhouse-java

It prompts a **lexical error**, yet the execution succeeds.

Open
#2,839 2 comments 0 reactions 0 assignees View on GitHub
area:sql-parser jdbc-v2
Dominant language
Java
Stars
1.6k
Forks
636
Avg merge
2d 23h
Merged PRs (30d)
29

Description

## Description

### Steps to reproduce
1.
2.
3.
### Error Log or Exception StackTrace

```
**This SQL can be executed successfully, but the following warning will appear in the com.clickhouse.jdbc.parser.ClickHouseSqlParser class.**

Lexical error at line 3, column 167. Encountered: "\u538b" (21387), after : "". If you believe the SQL is valid, please feel free to open an issue on Github with this warning and the following SQL attached.
INSERT INTO account_transaction_flow(id,tenant_id,out_tenant_id,name,customer_id,fund_account_id,customer_account_id,entry_status,transaction_date,revenue_type,revenue_amount,payment_id,expense_type,expense_amount,order_id,account_balance,remark,object_describe_id,object_describe_api_name,owner,out_owner,created_by,last_modified_by,create_time,last_modified_time,is_deleted,lock_status,life_status,record_type,extend_obj_data_id,life_status_before_invalid,lock_user,lock_rule,relevant_team,package,version,data_auth_code,out_data_auth_code,data_auth_id,out_data_auth_id,data_own_department,change_type,order_by,data_own_organization,dimension_d1,dimension_d2,dimension_d3,mc_currency,mc_exchange_rate,mc_functional_currency,mc_exchange_rate_version,outcome_object_api_name,outcome_object_data_id,partner_id,out_resources,relate_object_api_name,relate_object_data_id,display_name,origin_source,sys_modified_time,access_module,out_data_own_organization,out_data_own_department,revenue_source,bi_sys_flag,bi_sys_batch_id,bi_sys_is_deleted,bi_sys_version,bi_sys_ods_part,base_revenue_amount,base_expense_amount,base_account_balance) SETTINGS async_insert = 1, wait_for_async_insert = 1, async_insert_busy_timeout_ms = 3000 FORMAT CSVWithNames
id,tenant_id,out_tenant_id,name,customer_id,fund_account_id,customer_account_id,entry_status,transaction_date,revenue_type,revenue_amount,payment_id,expense_type,expense_amount,order_id,account_balance,remark,object_describe_id,object_describe_api_name,owner,out_owner,created_by,last_modified_by,create_time,last_modified_time,is_deleted,lock_status,life_status,record_type,extend_obj_data_id,life_status_before_invalid,lock_user,lock_rule,relevant_team,package,version,data_auth_code,out_data_auth_code,data_auth_id,out_data_auth_id,data_own_department,change_type,order_by,data_own_organization,dimension_d1,dimension_d2,dimension_d3,mc_currency,mc_exchange_rate,mc_functional_currency,mc_exchange_rate_version,outcome_object_api_name,outcome_object_data_id,partner_id,out_resources,relate_object_api_name,relate_object_data_id,display_name,origin_source,sys_modified_time,access_module,out_data_own_organization,out_data_own_department,revenue_source,bi_sys_flag,bi_sys_batch_id,bi_sys_is_deleted,bi_sys_version,bi_sys_ods_part,base_revenue_amount,base_expense_amount,base_account_balance
69645ab8c43a9a0007d6280f,40001402,,ATF20260112-241460,67ea57a160962d00010c35a0,67e6401e99eb000001882f73,67eab53027ca5a00013cb272,1,1768184505024,2,724,,,,,1195162.00,压测数据 - 20260112,620ceeecc5d11200016c4ce9,AccountTransactionFlowObj,1000,,1000,1000,1768184505308,1776672645764,-2,0,normal,income_record_type__c,,,,,,CRM,3,0364b60436865e26,9a1087f6420f7eab,255473,215003,999998,,,,,,,,,,,,,,,,,,,,1,,,,1,,,,i,,,
69645abac43a9a0007d64935,40001402,,ATF20260112-241482,67ea57c060962d00010c4b93,67e6401e99eb000001882f73,67eab52d99eb000001b07eaa,1,1768184506520,2,448,,,,,1200700.00,压测数据 - 20260112,620ceeecc5d11200016c4ce9,AccountTransactionFlowObj,1000,,1000,1000,1768184506629,1776672645629,-2,0,normal,income_record_type__c,,,,,,CRM,3,0364b60436865e26,9a1087f6420f7eab,255473,215003,999998,,,,,,,,,,,,,,,,,,,,1,,,,1,,,,i,,,
69645abac43a9a0007d650b2,40001402,,ATF20260112-241484,67ea57c35964de0001d2aa8e,67e6401e99eb000001882f73,67eab53599eb000001b0e9b2,1,1768184506817,2,944,,,,,1182001.00,压测数据 - 20260112,620ceeecc5d11200016c4ce9,AccountTransactionFlowObj,1000,,1000,1000,1768184507010,1776672645660,-2,0,normal,income_record_type__c,,,,,,CRM,3,0364b60436865e26,9a1087f6420f7eab,255473,215003,999998,,,,,,,,,,,,,,,,,,,,1,,,,1,,,,i,,,
69645abb9423d800079c0383,40001402,,ATF20260112-241496,67ea57d960962d00010c5dbf,67e6401e99eb000001882f73,67eab52f99eb000001b093c2,1,1768184507668,2,451,,,,,1205408.00,压测数据 - 20260112,620ceeecc5d11200016c4ce9,AccountTransactionFlowObj,1000,,1000,1000,1768184507592,1776672645729,-2,0,normal,income_record_type__c,,,,,,CRM,3,0364b60436865e26,9a1087f6420f7eab,255473,215003,999998,,,,,,,,,,,,,,,,,,,,1,,,,1,,,,i,,,

### Question

This SQL can be executed successfully, but the following warning will appear in the com.clickhouse.jdbc.parser.ClickHouseSqlParser class.
```

### Expected Behaviour

### Code Example

```java
public void insertDataIntoCH(String chUrl,
String table,
String tenantId,
List columns,
String csvData,
boolean chAsync,
Integer size) {
try {
String sql = buildClickHouseInsertSql(chUrl, table, columns, chAsync);
boolean asyncEnabled = !GrayManager.notUseAsyncInsert(chUrl) && chAsync;
{
JdbcConnection dst = jdbcService.getBiCHConnection(tenantId);
String fullSql = sql + "\n" + csvData;
dst.executeUpdate(fullSql);
}
log.info("success insert ClickHouse: ch:{},table:{},nums:{}", chUrl, table, size);
} catch (SQLException e) {
log.error("error insert", e);
throw new RuntimeException("ClickHouse insert error", e);
}
}

```

### Configuration

#### Environment
* [x] Cloud
* Client version:
* Language version: jdk21
* OS: linux

#### ClickHouse Server
* ClickHouse Server version: 25.8.13.73
* ClickHouse Server non-default settings, if any:
* `CREATE TABLE` statements for tables involved:
* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the INSERT statement with Chinese CSV data and inspect com.clickhouse.jdbc.parser.ClickHouseSqlParser, where the lexical warning is reported. Trace how the SQL is parsed before execution and add or update coverage for this input. Done means the valid insert still executes without emitting the lexical error warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.