[Python] AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 6, Couldn't resolve host name
- Lingua principale
- C++
- Stelle
- 17.1k
- Fork
- 4.3k
- Merge medio
- 3g 23h
- PR unite (30g)
- 101
Descrizione
### Describe the bug, including details regarding any error messages, version, and platform.
Hi there!
We are using the PyArrow library to read files from an S3 bucket, and we're encountering an intermittent error:
`OSError: When reading information for key '' in bucket '': AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 6, Couldn't resolve host name`
Please note that this error doesn't occur consistently, and the S3 bucket path is valid.
The reference code we're using is as follows:
```
import pyarrow as pa
import pyarrow.json as pj
uri = "s3://my-bucket/my-prefix/foo.json"
fs, path = pa.fs.FileSystem.from_uri(uri)
with fs.open_input_file(path) as f:
tbl = pj.read_json(f)
```
Error Details:
```
2024-03-12T13:06:05.616-07:00 [7]: with fs.open_input_file(path) as f:
2024-03-12T13:06:05.616-07:00 [7]: File "pyarrow/_fs.pyx", line 780, in pyarrow._fs.FileSystem.open_input_file
2024-03-12T13:06:05.616-07:00 [7]: File "pyarrow/error.pxi", line 154, in pyarrow.lib.pyarrow_internal_check_status
2024-03-12T13:06:05.616-07:00 [7]: File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
2024-03-12T13:06:05.616-07:00 [7]:OSError: When reading information for key '' in bucket '': AWS Error NETWORK_CONNECTION during HeadObject operation: curlCode: 6, Couldn't resolve host name
```
Could you please provide any suggestions on how to handle such intermittent network connectivity errors while reading from S3?
### Component(s)
Parquet, Python
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia con il riproduttore Python fornito e le voci del traceback in pyarrow/_fs.pyx e pyarrow/error.pxi, concentrandoti su FileSystem.from_uri e open_input_file durante la chiamata S3 HeadObject. Poiché la segnalazione richiede suggerimenti invece di una modifica definita, conferma il comportamento previsto per i retry o la gestione degli errori e definisci il completamento come una guida riproducibile o una correzione per il fallimento intermittente curlCode 6.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- aws, python
- Ambito
- cloud, data
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 25/100