activeloopai / activeloopai/deeplake

S3: GetBucketLocation required despite explicit aws_region, and the raised error names the wrong cause

Aberta
#3,163 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
C++
Estrelas
9.2k
Forks
722
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

**Version:** 4.7.0 · Linux aarch64, Python 3.12 · S3 storage

Two small things, same failure.

## 1. `s3:GetBucketLocation` is required even when the region is supplied

```python
creds = {"aws_access_key_id": ..., "aws_secret_access_key": ...,
"aws_session_token": ..., "aws_region": "us-east-1"} # region given
deeplake.create(url, creds=creds, schema={...})
```

```
[S3] Failed to get bucket region for URL: // with error:
[S3] Access denied: User: ... is not authorized to perform:
s3:GetBucketLocation on resource: "arn:aws:s3:::"
```

If the caller states the region, the lookup should be skipped. It is an extra permission on the bucket resource (not the object prefix), which means a least-privilege policy has to grant a bucket-level action purely to satisfy a call that was not needed.

## 2. The exception names the wrong cause

The message above goes to **stderr**. What the caller catches is:

```
deeplake._deeplake.StorageAccessDenied:
[S3] Access denied: _deeplake_log/_meta/0000000001c00000000.jsonl No response body.
```

So a program sees "cannot read `_deeplake_log/_meta/…`" when the actual denial was `s3:GetBucketLocation` on the bucket. Debugging a policy from that message sends you to the wrong statement — we first widened object permissions, twice, before finding the real one on stderr.

`No response body` is also misleading: there was a response, it was a 403.

## Ask

1. Skip the region lookup when `aws_region` is supplied.
2. Carry the underlying S3 error — action, resource, status — in the raised exception, not only on stderr.

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.