apache / apache/iceberg-python

[Bug] PyArrowFileIO fails to propagate s3.ssl.ca-cert to pyarrow.fs.S3FileSystem tls_ca_file_path

Abierto Apto para principiantes
#3,866 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
1.1k
Forks
581
Merge medio
1 d 17 h
PR fusionados (30 d)
77

Descripción

### Apache Iceberg version

0.11.0 (latest release)

### Please describe the bug 🐞

When configuring PyIceberg with a REST catalog (such as Apache Polaris) connected to an S3-compatible store using internal or self-signed certificates, providing "s3.ssl.ca-cert" in the catalog configuration does not take effect for PyArrow file operations.

PyIceberg's PyArrowFileIO instantiates pyarrow.fs.S3FileSystem without forwarding the custom CA certificate path to PyArrow's tls_ca_file_path argument. As a result, operations that write or read Parquet files fail with curlCode: 60 (SSL verification failure in PyArrow's underlying AWS C++ SDK / libcurl engine).

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Start at PyIceberg's PyArrowFileIO entry point where pyarrow.fs.S3FileSystem is instantiated, then trace how the catalog's "s3.ssl.ca-cert" setting is read. Verify the relevant PyArrow argument is covered for both Parquet reads and writes, and reproduce the issue with an internal or self-signed certificate to confirm the SSL verification failure is resolved.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
aws, python
Área
cloud, data-engineering
Tipo de issue
Error
Dificultad
2/5
Tiempo estimado
1-3 horas
Estado de actividad
Activo
Claridad
Bien especificado
Aptitud para principiantes
76/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.