ClickHouse / ClickHouse/clickhouse-odbc

`SQL_NULL_DATA` is ignored when a value buffer is bound (an empty string is sent); a parameter array stops after set 1 (`SQLMoreResults` executes it but returns `SQL_NO_DATA`)

Open
#582 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
285
Forks
105
Avg merge
3h 56m
Merged PRs (30d)
3

Description

Two parameter-binding defects, each with a plain-ODBC reproduction in C (unixODBC, `gcc -O1 -Wall x.c -lodbc`). Driver 1.5.5.20260810 (`SQL_DRIVER_VER 1.5.5.20260810`, Unicode build, Linux x86_64), server 26.7.5.10 over HTTP (`Url=http://127.0.0.1:8123`). Both reproduce identically on `ENGINE = Memory` and `MergeTree`.

### 1. `SQL_NULL_DATA` is ignored whenever `ParameterValuePtr` is non-NULL

ODBC says the length/indicator value decides: `SQL_NULL_DATA` means the parameter is NULL regardless of what the value buffer holds. The driver only honours it when the value pointer itself is NULL. With a buffer bound — which is what every application that reuses a bound buffer across rows does — it sends an empty string instead:

| bind | indicator | into `Nullable(String)` | into `Nullable(Int32)` |
|---|---|---|---|
| `ParameterValuePtr = NULL` | `SQL_NULL_DATA` | NULL ✓ | NULL ✓ |
| `ParameterValuePtr = buf` | `SQL_NULL_DATA` | **`''` stored, `SQL_SUCCESS`** | **`SQL_ERROR`: `Code: 32 ... Attempt to read after eof: value cannot be parsed as Nullable(Int32) for query parameter 'odbc_positional_1'`** |
| `ParameterValuePtr = buf` | `SQL_DEFAULT_PARAM` | `''` stored, `SQL_SUCCESS` | same error |

Into `Nullable(DateTime64)` the empty string becomes `1970-01-01 00:00:00` under `SQL_SUCCESS`; `Float64`, `Date`, `Decimal` and `Bool` fail server-side like `Int32`. The `String` and `DateTime64` cases are silent data corruption.

```c
#include
#include
#include
#include
static SQLHENV env; static SQLHDBC dbc;
static void diag(SQLSMALLINT t, SQLHANDLE h) {
SQLCHAR st[6], msg[1024]; SQLINTEGER ne; SQLSMALLINT len; SQLSMALLINT i = 1;
while (SQLGetDiagRec(t, h, i++, st, &ne, msg, sizeof msg, &len) == SQL_SUCCESS) printf(" %s (%d): %s\n", st, (int)ne, msg);
}
static void insert_null(const char* tag, SQLPOINTER ptr, SQLLEN ind) {
SQLHSTMT s; SQLAllocHandle(SQL_HANDLE_STMT, dbc, &s);
char sql[128]; snprintf(sql, sizeof sql, "INSERT INTO t_null (tag, s) VALUES ('%s', ?)", tag);
SQLPrepare(s, (SQLCHAR*)sql, SQL_NTS);
static SQLLEN indicator; indicator = ind;
SQLBindParameter(s, 1, SQL_PARAM_INPUT, SQL_C_CHAR, SQL_VARCHAR, 50, 0, ptr, 16, &indicator);
SQLRETURN rc = SQLExecute(s);
printf("%-12s ptr=%-4s ind=%ld -> rc=%d\n", tag, ptr ? "buf" : "NULL", (long)ind, (int)rc);
if (!SQL_SUCCEEDED(rc)) diag(SQL_HANDLE_STMT, s);
SQLFreeHandle(SQL_HANDLE_STMT, s);
}
int main(int argc, char** argv) {
const char* conn = argc > 1 ? argv[1] : "Driver=ClickHouse ODBC Driver (Unicode);Url=http://127.0.0.1:8123;Database=default;";
SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &env);
SQLSetEnvAttr(env, SQL_ATTR_ODBC_VERSION, (SQLPOINTER)SQL_OV_ODBC3, 0);
SQLAllocHandle(SQL_HANDLE_DBC, env, &dbc);
if (!SQL_SUCCEEDED(SQLDriverConnect(dbc, NULL, (SQLCHAR*)conn, SQL_NTS, NULL, 0, NULL, SQL_DRIVER_NOPROMPT))) { diag(SQL_HANDLE_DBC, dbc); return 1; }
SQLHSTMT s; SQLAllocHandle(SQL_HANDLE_STMT, dbc, &s);
SQLExecDirect(s, (SQLCHAR*)"DROP TABLE IF EXISTS t_null", SQL_NTS);
SQLExecDirect(s, (SQLCHAR*)"CREATE TABLE t_null (tag String, s Nullable(String)) ENGINE = Memory", SQL_NTS);
static char buf[16] = "hello";
insert_null("nullptr", NULL, SQL_NULL_DATA);
insert_null("buf_null", buf, SQL_NULL_DATA);
insert_null("buf_default", buf, SQL_DEFAULT_PARAM);
SQLExecDirect(s, (SQLCHAR*)"SELECT tag, isNull(s), s = '' FROM t_null ORDER BY tag", SQL_NTS);
char tag[32], isnull[8], empty[8]; SQLLEN i1, i2, i3;
SQLBindCol(s, 1, SQL_C_CHAR, tag, sizeof tag, &i1); SQLBindCol(s, 2, SQL_C_CHAR, isnull, sizeof isnull, &i2); SQLBindCol(s, 3, SQL_C_CHAR, empty, sizeof empty, &i3);
while (SQLFetch(s) == SQL_SUCCESS) printf("stored: %-12s isNull=%s s=''=%s\n", tag, isnull, i3 == SQL_NULL_DATA ? "NULL" : empty);
SQLFreeHandle(SQL_HANDLE_STMT, s); SQLDisconnect(dbc); SQLFreeHandle(SQL_HANDLE_DBC, dbc); SQLFreeHandle(SQL_HANDLE_ENV, env); return 0;
}
```

Output:

```
nullptr ptr=NULL ind=-1 -> rc=0
buf_null ptr=buf ind=-1 -> rc=0
buf_default ptr=buf ind=-5 -> rc=0
stored: buf_default isNull=0 s=''=1
stored: buf_null isNull=0 s=''=1
stored: nullptr isNull=1 s=''=NULL
```

Expected: all three rows stored as NULL. `SQLGetInfo(SQL_DESCRIBE_PARAMETER)` is `N` and `SQLDescribeParam` answers `SQL_UNKNOWN_TYPE`, so an application cannot even work around it by describing the parameter first; the only working shape is a NULL value pointer.

### 2. A parameter array stops after set 1: `SQLMoreResults` executes the next set but returns `SQL_NO_DATA`

#324 / #325 established this driver's protocol for `SQL_ATTR_PARAMSET_SIZE > 1`: `SQLExecute` sends parameter set 0, and each `SQLMoreResults` sends the next set (`Statement::requestNextPackOfResultSets`, `++next_param_set_idx`). That is not what the ODBC specification describes (a single `SQLExecute` processes every set; `SQLMoreResults` is for multiple result sets), but it is the documented way to use arrays here, and per #324 it worked in 1.1.9.

On 1.5.5.20260810 it does not: the first `SQLMoreResults` executes set 1 and updates `SQL_ATTR_PARAMS_PROCESSED_PTR` to 1 — and returns `SQL_NO_DATA` at the same time, because `advanceToNextResultSet()` reports whether a result set came back and an `INSERT` never has one. Every later `SQLMoreResults` also returns `SQL_NO_DATA` and executes nothing. So a 5-set array inserts exactly 2 rows, with `SQL_SUCCESS` from `SQLExecute` and `SQL_NO_DATA` from `SQLMoreResults`, i.e. the sequence a conforming caller reads as "all done". Column-wise and row-wise binding, `Memory` and `MergeTree`, with or without a NULL in a later set: identical.

```c
#include
#include
#include
#include
#define N 5
static SQLHENV env; static SQLHDBC dbc;
static void diag(SQLSMALLINT t, SQLHANDLE h) {
SQLCHAR st[6], msg[1024]; SQLINTEGER ne; SQLSMALLINT len; SQLSMALLINT i = 1;
while (SQLGetDiagRec(t, h, i++, st, &ne, msg, sizeof msg, &len) == SQL_SUCCESS) printf(" %s (%d): %s\n", st, (int)ne, msg);
}
int main(int argc, char** argv) {
const char* conn = argc > 1 ? argv[1] : "Driver=ClickHouse ODBC Driver (Unicode);Url=http://127.0.0.1:8123;Database=default;";
SQLAllocHandle(SQL_HANDLE_ENV, SQL_NULL_HANDLE, &env);
SQLSetEnvAttr(env, SQL_ATTR_ODBC_VERSION, (SQLPOINTER)SQL_OV_ODBC3, 0);
SQLAllocHandle(SQL_HANDLE_DBC, env, &dbc);
if (!SQL_SUCCEEDED(SQLDriverConnect(dbc, NULL, (SQLCHAR*)conn, SQL_NTS, NULL, 0, NULL, SQL_DRIVER_NOPROMPT))) { diag(SQL_HANDLE_DBC, dbc); return 1; }
SQLHSTMT s; SQLAllocHandle(SQL_HANDLE_STMT, dbc, &s);
SQLExecDirect(s, (SQLCHAR*)"DROP TABLE IF EXISTS t_arr", SQL_NTS);
SQLExecDirect(s, (SQLCHAR*)"CREATE TABLE t_arr (i Int32, s String) ENGINE = Memory", SQL_NTS);
static SQLINTEGER iv[N]; static char sv[N][16]; static SQLLEN ii[N], si[N];
for (int k = 0; k < N; k++) { iv[k] = 100 + k; snprintf(sv[k], 16, "s%d", k); ii[k] = sizeof(SQLINTEGER); si[k] = SQL_NTS; }
SQLULEN processed = 987654321; SQLUSMALLINT status[N]; for (int k = 0; k < N; k++) status[k] = 99;
SQLSetStmtAttr(s, SQL_ATTR_PARAM_BIND_TYPE, (SQLPOINTER)SQL_PARAM_BIND_BY_COLUMN, 0);
SQLSetStmtAttr(s, SQL_ATTR_PARAMSET_SIZE, (SQLPOINTER)(SQLULEN)N, 0);
SQLSetStmtAttr(s, SQL_ATTR_PARAMS_PROCESSED_PTR, &processed, 0);
SQLSetStmtAttr(s, SQL_ATTR_PARAM_STATUS_PTR, status, 0);
SQLPrepare(s, (SQLCHAR*)"INSERT INTO t_arr (i, s) VALUES (?, ?)", SQL_NTS);
SQLBindParameter(s, 1, SQL_PARAM_INPUT, SQL_C_SLONG, SQL_INTEGER, 10, 0, iv, sizeof(SQLINTEGER), ii);
SQLBindParameter(s, 2, SQL_PARAM_INPUT, SQL_C_CHAR, SQL_VARCHAR, 15, 0, sv, 16, si);
SQLRETURN rc = SQLExecute(s);
printf("SQLExecute rc=%d processed=%lu\n", (int)rc, (unsigned long)processed); if (!SQL_SUCCEEDED(rc)) diag(SQL_HANDLE_STMT, s);
for (int k = 1; k < N; k++) { /* the #324 protocol: one SQLMoreResults per remaining set, kept going regardless of SQL_NO_DATA */
rc = SQLMoreResults(s);
printf("SQLMoreResults rc=%d processed=%lu status=", (int)rc, (unsigned long)processed);
for (int j = 0; j < N; j++) printf("%u ", status[j]);
printf("(99 = untouched)\n");
if (rc != SQL_NO_DATA && !SQL_SUCCEEDED(rc)) diag(SQL_HANDLE_STMT, s);
}
SQLFreeStmt(s, SQL_RESET_PARAMS); SQLSetStmtAttr(s, SQL_ATTR_PARAMSET_SIZE, (SQLPOINTER)1, 0);
SQLExecDirect(s, (SQLCHAR*)"SELECT count(), groupArray(i) FROM t_arr", SQL_NTS);
char cnt[16], arr[256]; SQLLEN c1, c2; SQLBindCol(s, 1, SQL_C_CHAR, cnt, sizeof cnt, &c1); SQLBindCol(s, 2, SQL_C_CHAR, arr, sizeof arr, &c2);
if (SQLFetch(s) == SQL_SUCCESS) printf("rows in table = %s %s (expected %d)\n", cnt, arr, N);
SQLFreeHandle(SQL_HANDLE_STMT, s); SQLDisconnect(dbc); SQLFreeHandle(SQL_HANDLE_DBC, dbc); SQLFreeHandle(SQL_HANDLE_ENV, env); return 0;
}
```

Output:

```
SQLExecute rc=0 processed=0
SQLMoreResults rc=100 processed=1 status=0 0 99 99 99 (99 = untouched)
SQLMoreResults rc=100 processed=1 status=0 0 99 99 99 (99 = untouched)
SQLMoreResults rc=100 processed=1 status=0 0 99 99 99 (99 = untouched)
SQLMoreResults rc=100 processed=1 status=0 0 99 99 99 (99 = untouched)
rows in table = 2 [100,101] (expected 5)
```

Two smaller things visible in the same output: `SQL_ATTR_PARAMS_PROCESSED_PTR` holds the index of the set being sent rather than the number of sets completed (0 after `SQLExecute` has inserted set 0 — the `TODO` in `requestNextPackOfResultSets` says as much), and a plain `SQLExecute` with no `SQLMoreResults` at all leaves N−1 sets unexecuted under `SQL_SUCCESS`, which is what an application written to the specification does. A multi-row `INSERT ... VALUES (?,?),(?,?),...` with one scalar parameter per slot does insert every row, so that is the workaround available today (the placeholder count is bounded by the server's HTTP form-field limit, ~1,000).

`SQLRowCount` answering 0 for every write is #335, so I have not repeated it here.

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied plain-ODBC C reproductions, then inspect the parameter-array path around Statement::requestNextPackOfResultSets and advanceToNextResultSet. Done means SQL_NULL_DATA produces NULL with a bound buffer, and every parameter-array set executes with return values and processed counts that let a conforming caller continue through all sets.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, sql
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.