crate / crate/cratedb-toolkit

`ctk cfr sys-import` fails to import `segments` table (and others)

Open
#880 0 comments 0 reactions 1 assignee Claimed by @florinutz View on GitHub
bug
Dominant language
Python
Stars
10
Forks
5
Avg merge
7d 5h
Merged PRs (30d)
7

Description

## Description

Running `ctk cfr sys-import` also imports data for the `sys.segments` table. That table as an `attributes` column of type `OBJECT` containing values such as `{"attributes":{"Lucene90StoredFieldsFormat.mode":"BEST_SPEED"}, [...]}`.
Because CrateDB doesn't allow dots in column names, importing such data silently fails. The `sys-segments` table is empty after the `ctk cfr sys-import` run.

## Reproduction

1. Ensure you have some data in your CrateDB: `CREATE TABLE t1 (a INTEGER);`
2. To generate segments, there needs to be data in the table: `INSERT INTO t1 VALUES (1), (2), (3);`
3. Force segments to be written: `REFRESH TABLE t1;`
4. Configure that cluster to be used by the toolkit: `export CRATEDB_CLUSTER_URL=http://crate:password@localhost:4200/`
5. Create an export: `ctk cfr sys-export debug`
6. Import it: `ctk cfr sys-import debug/crate/2026-08-06T14-23-55/sys`
7. Check `SELECT * FROM "sys-sessions"` (it's empty)

Optionally, you can try editing `debug/crate/2026-08-06T14-23-55/sys/data/sys-segments.jsonl` and replace `Lucene90StoredFieldsFormat.mode` with something like `Lucene90StoredFieldsFormat_mode`. The import then succeeds.

## Potential solutions

1. Use `OBJECT(IGNORED)` in the schema. The limitation of dots in identifiers seems only to apply to indexed columns, i.e. `OBJECT(DYNAMIC)`.
2. Use `TEXT` in the schema, but then its less comfortable to query
3. Apply some renaming of problematic values, although that would make investigations difficult if the data doesn't match 1:1 between expoert and import clusters

---

While writing this down, I noticed that other tables are affected as well, such as:
* `sys.users`: There can also be `settings['memory.operation_limit']` (if the setting is used)
* `sys.sessions`: Can also have `settings['memory.operation_limit']`.
* `sys.cluster`: `state` (`TEXT` column) can have values that are too large for indexing (limit is 32766 bytes, see https://cratedb.com/docs/crate/reference/en/latest/general/ddl/storage.html). Should have `INDEX OFF STORAGE WITH (columnstore = false)`.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.