Support wildcard expression to load multiple files from GCS
- Linguagem predominante
- Python
- Estrelas
- 22.1k
- Forks
- 5.4k
- Merge médio
- 5h
- PRs com merge (30d)
- 671
Descrição
## Tell us about the problem you're trying to solve
We want to load files on GCS with wildcard expression, say about `gs://my-bucket/folder/data/data*.csv`, because it is quite common to use multiple files to express single data.
## Describe the solution you’d like
It seems that `URLFile` seems to accept only a single file. I am not sure the best solution. We might have to change the specification to deal with file(s).
https://github.com/airbytehq/airbyte/blob/d4267818ae0f81f584f856b6b208ee76d774e15a/airbyte-integrations/connectors/source-file/source_file/client.py#L51
## Describe the alternative you’ve considered or used
Technically, we might be able to read files one by one. But, it is very annoying.
┆Issue is synchronized with this [Asana task](https://app.asana.com/0/1200367912513076/1200368096116135) by [Unito](https://www.unito.io)
Guia de contribuição
Avaliação
Esta issue ainda não foi avaliada.