Reading external delta table registered in unity catalog doesn't work
- Dominant language
- C++
- Stars
- 110
- Forks
- 44
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 4
Description
Thanks for building this extension. I've found some scenarios where DuckDB's performance seems pretty magical, so I'm keen to use DuckDB if I can, but I'm currently having issues with unity catalog.
Created a simple external table in our Azure Databricks unity catalog
Querying this with other delta table clients works fine.
Attempt to read this using DuckDB CLI:
Install extensions, setup auth and attach a unity catalog
```
INSTALL delta;
LOAD delta;
INSTALL azure;
LOAD azure;
INSTALL unity_catalog;
LOAD unity_catalog;
CREATE SECRET (
TYPE azure,
PROVIDER credential_chain,
CHAIN cli
);
CREATE SECRET (
TYPE unity_catalog,
TOKEN ,
ENDPOINT
);
ATTACH 'dev_user' as dev_user (TYPE UC_CATALOG);
```
NOTE: we shouldn't need to configure the `azure` secret. It should be using unity catalog token federation, but that is a [separate issue](https://github.com/duckdb/unity_catalog/issues/36).
Test that unity catalog is working to some extent
```
SHOW TABLES FROM dev_user;
```
Returns a correct list of tables
Attempt to read an external table
```
SELECT * FROM dev_user.tomnewton.external_unity_catalog;
```
```
Binder Error:
SELECT list is empty after resolving * expressions!
```
I'm testing with duckdb 1.5.5 using the python client and the CLI. Attempting to connect to unity catalog in an Azure managed Databrticks using unity_catalog extension version fd85147
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the DuckDB CLI reproduction using the delta, azure, and unity_catalog extensions, then verify that SHOW TABLES FROM dev_user works before querying the external table. Trace the unity_catalog external-table path and compare it with the working table listing; done means SELECT * FROM dev_user.tomnewton.external_unity_catalog returns the table columns and rows instead of an empty SELECT list error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, cpp
- Domain
- cloud, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100