Support wildcard expression to load multiple files from GCS
- Lingua principale
- Python
- Stelle
- 22.1k
- Fork
- 5.3k
- Metriche di merge delle PR
- Metriche PR in attesa
Descrizione
## Tell us about the problem you're trying to solve
We want to load files on GCS with wildcard expression, say about `gs://my-bucket/folder/data/data*.csv`, because it is quite common to use multiple files to express single data.
## Describe the solution you’d like
It seems that `URLFile` seems to accept only a single file. I am not sure the best solution. We might have to change the specification to deal with file(s).
https://github.com/airbytehq/airbyte/blob/d4267818ae0f81f584f856b6b208ee76d774e15a/airbyte-integrations/connectors/source-file/source_file/client.py#L51
## Describe the alternative you’ve considered or used
Technically, we might be able to read files one by one. But, it is very annoying.
┆Issue is synchronized with this [Asana task](https://app.asana.com/0/1200367912513076/1200368096116135) by [Unito](https://www.unito.io)
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.