crate / crate/crate

COPY TO S3 with invalid URL takes a very long time to fail

Open
#18,102 3 comments 0 reactions 0 assignees View on GitHub
triage
Dominant language
Java
Stars
4.4k
Forks
616
Avg merge
14h 49m
Merged PRs (30d)
206

Description

### CrateDB version

5.10.10

### CrateDB setup information

Single node and multi-node (see note about multi-node in the Actual Result section)

### Problem description

Trying to export data to S3 with `COPY TO` it can happen that one can make a mistake on the URL, when this happens CrateDB takes a very long time to report it which leads to the impression it is working on the export making so an admin will only notice the mistake much later.

### Steps to Reproduce

```sql
CREATE TABLE tbl1 (a integer);
INSERT INTO tbl1 SELECT 1;
REFRESH TABLE tbl1;
COPY tbl1 TO DIRECTORY 's3://invalid:invalid@s3.amazonaws.com:80/nonexisting/hernantest' with (protocol = 'https', compression = 'gzip',format='json_object',wait_for_completion='true');
```

### Actual Result

```
SdkClientException[Unable to execute HTTP request: Remote host terminated the handshake]
```
but After ~15 minutes (and it seem on multi-node clusters this delay time may be multiplied by the number of nodes)

### Expected Result

The same message but within a minute or so.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.