The task fingerprint-fields fail if table doesn't exist or query failed
- Dominant language
- Clojure
- Stars
- 49.3k
- Forks
- 6.8k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 653
Description
### Describe the bug
Hello!
Some time ago I created an [issue]() about a problem with the sync fields step. This behavior has been updated and now this step works fine. But another step (fingerprint-fields) fails if there are problems with the table and the sync request cannot be completed.
### To Reproduce
After the sync-fields step, break some table and watch fingerprint-fields break too.
### Expected behavior
The synchronization process returns an error or warning and continues synchronizing the structure of another table.
### Logs
The logs in the web interface contain the following error:
```
{"status":"failed","exception":"class clojure.lang.ExceptionInfo","message":null,"stacktrace":[null],"ex-data":{"toucan2/context-trace":[["execute SQL with class com.mchange.v2.c3p0.impl.NewProxyConnection",{"toucan2.jdbc.query/sql-args":["SELECT * FROM `metabase_table` WHERE ((`active` = TRUE) AND (`visibility_type` IS NULL)) AND (`db_id` = ?)",39]}],["resolve connection",{"toucan2.connection/connectable":"class metabase.db.connection.ApplicationDB"}],["resolve connection",{"toucan2.connection/connectable":"default"}],["resolve connection",{"toucan2.connection/connectable":null}],{"toucan2.pipeline/rf":"clojure.core$map$fn__5950$fn__5951@32ea7cf7"},["with compiled query",{"toucan2.pipeline/compiled-query":["SELECT * FROM `metabase_table` WHERE ((`active` = TRUE) AND (`visibility_type` IS NULL)) AND (`db_id` = ?)",39]}],["with built query",{"toucan2.pipeline/built-query":{"select":["*"],"from":[["metabase_table"]],"where":["and",["and",["=","active",true],["=","visibility_type",null]],null,null,["=","db_id",39]]}}],["with resolved query",{"toucan2.pipeline/resolved-query":{"where":["and",["and",["=","active",true],["=","visibility_type",null]],null,null]}}],["with parsed args",{"toucan2.pipeline/query-type":"toucan.query-type/select.instances","toucan2.pipeline/parsed-args":{"kv-args":{"db_id":39},"queryable":{"where":["and",["and",["=","active",true],["=","visibility_type",null]],null,null]}}}],["with model",{"toucan2.pipeline/model":"model/Table"}],["with unparsed args",{"toucan2.pipeline/query-type":"toucan.query-type/select.instances","toucan2.pipeline/unparsed-args":["model/Table","db_id",39,{"where":["and",["and",["=","active",true],["=","visibility_type",null]],null,null]}]}]]},"original-info":null}
```
There may be various errors in the disk logs that prevent the query from being executed:
```
java.sql.SQLException: Code: 307. DB::Exception: Limit for rows or bytes to read exceeded, max bytes: ** GiB, current bytes: ** GiB: While executing Remote. (TOO_MANY_BYTES)
```
or
```
com.clickhouse.client.api.ServerException: Code: 497. DB::Exception: user_name: Not enough privileges. To execute this query, it's necessary to have the grant SELECT(***) ON ***. (ACCESS_DENIED)
```
or
```
Code: 279. DB::Exception: All connection tries failed. Log: There is no table `db`.`table` on server: *** There is no table *** on server: ***: While executing Remote. (ALL_CONNECTION_TRIES_FAILED)
```
and so on.
### Information about your Metabase installation
```JSON
{
"browser-info": {
"platform": "MacIntel",
"userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36",
"vendor": "Google Inc."
},
"metabase-info": {
"databases": [
"exasol",
"mongo",
"clickhouse",
"mysql"
],
"run-mode": "prod",
"plan-alias": "",
"version": {
"date": "2025-05-27",
"tag": "v0.54.10",
"hash": "1f67bd2"
},
"hosting-env": "unknown",
"application-database": "mysql",
"application-database-details": {
"database": {
"name": "MariaDB",
"version": "10.5.13-MariaDB"
},
"jdbc-driver": {
"name": "MariaDB Connector/J",
"version": "2.7.10"
}
}
},
"system-info": {
"file.encoding": "UTF-8",
"java.runtime.name": "OpenJDK Runtime Environment",
"java.runtime.version": "21.0.7+6-LTS",
"java.vendor": "Eclipse Adoptium",
"java.vendor.url": "https://adoptium.net/",
"java.version": "21.0.7",
"java.vm.name": "OpenJDK 64-Bit Server VM",
"java.vm.version": "21.0.7+6-LTS",
"os.name": "Linux",
"os.version": "5.14.0-580.el9.x86_64",
"user.language": "en",
}
}
```
### Severity
blocking usage of clickhouse cluster
### Additional context
*No response*
Contributor guide
Research direction
Start with the fingerprint-fields step and compare its behavior with the preceding sync-fields step. Reproduce the issue using a missing, inaccessible, or failing ClickHouse table, then inspect the web and disk logs. Done means synchronization reports an error or warning for the affected table and continues processing other tables.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, clojure
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100