apache / apache/arrow

[Python] AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 6, Couldn't resolve host name

Aperta
#40,539 7 commenti 7 reazioni 0 assegnatari Vedi su GitHub
Component: Parquet Component: Python Type: bug
Lingua principale
C++
Stelle
17.1k
Fork
4.3k
Merge medio
3g 23h
PR unite (30g)
101

Descrizione

### Describe the bug, including details regarding any error messages, version, and platform.

Hi there!

We are using the PyArrow library to read files from an S3 bucket, and we're encountering an intermittent error:

`OSError: When reading information for key '' in bucket '': AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 6, Couldn't resolve host name`

Please note that this error doesn't occur consistently, and the S3 bucket path is valid.

The reference code we're using is as follows:
```
import pyarrow as pa
import pyarrow.json as pj

uri = "s3://my-bucket/my-prefix/foo.json"

fs, path = pa.fs.FileSystem.from_uri(uri)

with fs.open_input_file(path) as f:
tbl = pj.read_json(f)
```

Error Details:
```
2024-03-12T13:06:05.616-07:00 [7]: with fs.open_input_file(path) as f:

2024-03-12T13:06:05.616-07:00 [7]: File "pyarrow/_fs.pyx", line 780, in pyarrow._fs.FileSystem.open_input_file

2024-03-12T13:06:05.616-07:00 [7]: File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status

2024-03-12T13:06:05.616-07:00 [7]: File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status

2024-03-12T13:06:05.616-07:00 [7]:OSError: When reading information for key '' in bucket '': AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 6, Couldn't resolve host name

```

Could you please provide any suggestions on how to handle such intermittent network connectivity errors while reading from S3?

### Component(s)

Parquet, Python

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia con il riproduttore Python fornito e le voci del traceback in pyarrow/_fs.pyx e pyarrow/error.pxi, concentrandoti su FileSystem.from_uri e open_input_file durante la chiamata S3 HeadObject. Poiché la segnalazione richiede suggerimenti invece di una modifica definita, conferma il comportamento previsto per i retry o la gestione degli errori e definisci il completamento come una guida riproducibile o una correzione per il fallimento intermittente curlCode 6.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, python
Ambito
cloud, data
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.