duckdb / duckdb/unity_catalog

SELECT on a UC-attached table with a collated STRING column fails

Open
#112 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
110
Forks
44
Avg merge
2d 13h
Merged PRs (30d)
4

Description

## Summary

Attaching a Unity Catalog catalog via the `uc_catalog` extension and then
querying any table that has a `STRING COLLATE UTF8_BINARY` column fails with:

```
Not implemented Error: Tried to fallback to unknown type for 'string collate UTF8_BINARY'
```

Collated `STRING` columns are the current default for string columns created
by recent Databricks Runtime versions, so this affects essentially any table
created on a modern Databricks Runtime with Unity Catalog enabled — not an
edge case.

## Environment

Reproduced identically on two DuckDB versions:

- DuckDB `v1.5.5 (Variegata) d8cdaa33fd` (current, confirmed 2026-09-07)
- `unity_catalog` extension: version `fd85147`, installed from the
`core` repository (`REPOSITORY` install mode)
- `delta` extension: version `45c4087`, installed from the `core`
repository
- DuckDB `v2.1.0-alpha40575 (Unknown Version) 154c2c8d5f` (initial repro,
2026-09-03)
- `unity_catalog` extension: version `b593157d11`, installed from the
`core` repository
- `delta` extension: version `10a49159f8`, installed from the `core`
repository
- OS/arch: macOS, `osx_arm64`
- Unity Catalog server: a Databricks workspace's built-in Unity Catalog
metastore.

## How I connected

Using the `uc_catalog` extension against a live Databricks Unity Catalog
metastore (not the OSS Unity Catalog server), with a Databricks personal
access token obtained via the Databricks CLI:

```sql
INSTALL uc_catalog;
LOAD uc_catalog;
INSTALL delta;
LOAD delta;

CREATE SECRET uc_secret (
TYPE uc,
TOKEN '',
ENDPOINT 'https://XXX.databricks.com',
AWS_REGION 'us-east-1'
);

ATTACH 'users' AS users (TYPE uc_catalog, SECRET uc_secret, DEFAULT_SCHEMA 'default');

SELECT COUNT(*) FROM users.XXX.XXX;
```

The `ATTACH` itself succeeds. The failure happens on the subsequent
`SELECT` against a table containing a collated `STRING` column.

## Error

```
Not implemented Error: Tried to fallback to unknown type for 'string collate UTF8_BINARY'
```

Reproduced consistently against multiple tables in the same catalog/schema,
all created by a recent Databricks Runtime (where collation is now the
default for `STRING` columns)

## Root cause (isolated)

Reading the *same* table's underlying data directly off S3 with the `delta`
extension's `delta_scan()` — bypassing the `unity_catalog` extension
entirely — works with no error and returns the expected row count:

```sql
INSTALL delta;
LOAD delta;
CREATE SECRET s3_secret (TYPE s3, PROVIDER credential_chain, ...);
SELECT COUNT(*) FROM delta_scan('s3:///.../tables//');
-- 606
```

This shows the `delta` extension itself can read and decode the underlying
Parquet/Delta files for this table without issue. So the failure is
specific to the `unity_catalog` extension's `ATTACH` layer: when it maps
Unity Catalog's column type metadata to a DuckDB logical type, it doesn't
know how to represent (or fall back for) the collated-string type
(`STRING COLLATE UTF8_BINARY`), and errors out before the `delta` extension
ever gets a chance to scan the file.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the unity_catalog extension's ATTACH layer and its Unity Catalog column-type metadata mapping, focusing on handling for `STRING COLLATE UTF8_BINARY`. Use the supplied ATTACH and SELECT reproduction, with `delta_scan()` as the working comparison; done means SELECT succeeds and returns the expected row count for tables with collated STRING columns.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.