duckdb / duckdb/duckdb-httpfs

Issue: Cannot attach DuckDB database from S3 bucket despite proper credentials

Open
#32 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
60
Forks
100
Avg merge
1h 50m
Merged PRs (30d)
25

Description

### What happens?

I'm trying to attach a remote DuckDB database stored in an S3 bucket, but I'm encountering an error indicating the database doesn't exist, even though I have proper read/write access to the bucket.
Environment
DuckDB version: v1.2.0
Python version: 3.12
Operating System: macOS

Traceback (most recent call last):
File "/Users/badrou/repository/trendduck/exploration/13_read_remote_catalog.py", line 17, in
conn.sql(f"ATTACH DATABASE '{catalog_s3_path}' AS github")
duckdb.duckdb.IOException: IO Error: Cannot open database "https://billionrows-trendduck.s3.eu-west-3.amazonaws.com/catalog/github_catalog.ddb" in read-only mode: database does not exist

Additional Information
I can read and write to the S3 bucket without issues using the same credentials
The S3 path is correct and I've verified the bucket exists
I've tried with and without trailing slashes in the path
AWS credentials are properly loaded and working for other S3 operations

Expected Behavior
DuckDB should be able to attach the remote database from S3 without errors.

### To Reproduce

1. install and load the required extensions
2. Load AWS credentials
3. Try to attach a database from S3

```import duckdb

conn = duckdb.connect(":memory:")

# Load required extensions
conn.sql("INSTALL httpfs")
conn.execute("LOAD httpfs;")
conn.sql("LOAD aws")
conn.execute("CALL load_aws_credentials();")

# Try to attach the remote database
catalog_s3_path = "s3://billionrows-trendduck/catalog/github_catalog.ddb"
conn.sql(f"ATTACH DATABASE '{catalog_s3_path}' AS github") # doesn't work

conn.sql(" select * from read_parquet("s3://billionrows-trendduck/table/*.parquet") # Works
```

### OS:

arm64

### DuckDB Version:

v1.2.0

### DuckDB Client:

Python

### Hardware:

_No response_

### Full Name:

Erraji Badr

### Affiliation:

Data unboxed

### What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

### Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

### Did you include all code required to reproduce the issue?

- [x] Yes, I have

### Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

- [x] Yes, I have

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.