duckdb / duckdb/duckdb-python

Pushdown Filter Type BLOOM_FILTER is not currently supported in PyArrow Scans

Ouverte
#463 3 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
needs more info
Langage dominant
Python
Étoiles
187
Forks
112
Merge moyen
13 h 29 min
PR mergées (30 j)
17

Description

### What happens?

Hi,

I am running into this error message and am unsure what to do.
```
_duckdb.NotImplementedException: Not implemented Error: Pushdown Filter Type BLOOM_FILTER is not currently supported in PyArrow Scans
```
It seems I can either disable pushdown filtering through joins, or materialize the query before doing my join. I would rather not do either, since it is a lot of data that I am processing.

Reading the source files with a non-pyarrow reader is unfortunately not an option at this time.

### To Reproduce

Unfortunately, I don't really have a reproducer, but I think the maintainers will know what the issue is anyways from the error message. It seems to me that Pyarrow do support bloom filter, but the pushdown is just not implemented?

### OS:

Ubuntu 22

### DuckDB Package Version:

1.5.2

### Python Version:

3.11

### Full Name:

Jakob Stigenberg

### Affiliation:

Qubos Systematic

### What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

### Did you include all relevant data sets for reproducing the issue?

No - I cannot share the data sets because they are confidential

### Did you include all code required to reproduce the issue?

- [ ] Yes, I have

### Did you include all relevant configuration to reproduce the issue?

- [x] Yes, I have

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez par le chemin d’analyse PyArrow qui émet la BLOOM_FILTER NotImplementedException et examinez comment le pushdown du filtre de jointure y parvient. Le rapport ne contient ni jeu de données ni code reproductible ; établissez donc d’abord un cas minimal avec DuckDB 1.5.2 et Python 3.11. Le travail est considéré comme terminé lorsque l’analyse PyArrow prise en charge ne lève plus cette erreur pour le scénario de pushdown signalé, avec un comportement validé à l’aide d’un cas de régression approprié.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python
Domaine
databases
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Calme
Clarté
À clarifier
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.