duckdb / duckdb/duckdb-httpfs

Attempted to register an already registered secret type 's3'

Open
#30 5 comments 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
60
Forks
100
Avg merge
1h 50m
Merged PRs (30d)
25

Description

### What happens?

Parallel processing of adding temporary secrets with unique names results in errors.

**Full error message:**

```
java.sql.SQLException: Extension Autoloading Error: An error occurred while trying to automatically install the required extension 'httpfs':
Initialization function "httpfs_init" from file "/root/.duckdb/extensions/v0.10.0/linux_amd64_gcc4/httpfs.duckdb_extension" threw an exception: "Attempted to register an already registered secret type: 's3'"
```

**DuckDB Code:**
The error is being thrown from here:
https://github.com/duckdb/duckdb/blob/main/src/main/secret/secret_manager.cpp#L100-L107

Is there an issue with synchronization or locking here?

### To Reproduce

SQL:
```sql
CREATE OR REPLACE TEMPORARY SECRET s_367ea2 (
TYPE S3,
KEY_ID 'accessKey',
SECRET 'secretKey',
SESSION_TOKEN 'sessionToken',
REGION 'us-east-1',
ENDPOINT 's3.us-east-1.amazonaws.com',
SCOPE 's3://bucket/path/to/report.parquet'
);
```

JAVA:
```java
// Thread pool setup
final ExecutorService threadPool = Executors.newFixedThreadPool(5);

// Create connection properties
final Properties connectionProperties = new Properties();
connectionProperties.setProperty("access_mode", "READ_WRITE");

// Create workloads on the executor
final List> completableFutures = IntStream.range(1, 6)
.mapToObj(taskNumber -> CompletableFuture.runAsync(() -> {
try (final Connection connection = DriverManager.getConnection("jdbc:duckdb:", connectionProperties);
final Statement statement = connection.createStatement()) {
statement.execute("");
}
}, threadPool))
.toList();

// Wait for all to complete
final CompletableFuture.allOf(completableFutures.toArray(CompletableFuture[]::new))
.whenComplete((v, e) -> threadPool.shutdown())
.join();
```

### OS:

Linux (amd64)

### DuckDB Version:

0.10.0

### DuckDB Client:

Java (JDBC)

### Full Name:

Brian Wyka

### Affiliation:

LogicMonitor

### Have you tried this on the latest [nightly build](https://duckdb.org/docs/installation/?version=main)?

I have **NOT** tested with a nightly build

### Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

- [X] Yes, I have

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.