GCS support for FdReader used in (downstream) TFDS
- Dominant language
- C++
- Stars
- 495
- Forks
- 66
- PR merge metrics
- No merged PRs in 30d
Description
Following up on [google/array_record/issues/120](https://github.com/google/array_record/issues/120)
The following code fails when it shouldn't
```python
import tensorflow_datasets as tfds
ds = tfds.data_source("ref_coco", data_dir="gs://ref_coco", try_gcs=True)
next(iter(ds['train']))
```
[Colab link](https://colab.research.google.com/drive/1iezVDZBJrWtP3fVTpAMNGb6qfENZNKuD)
The path gets handed over to a Riegeli `FdReader` ([source](https://github.com/google/array_record/blob/1f72e6d22336705df5079e2089e15a8208cdf351/python/array_record_module.cc#L94)) and the `array_record` maintainers have pointed me upstream. I can't find GCS support mentioned in the docs, but given TFDS provides a `try_gcs` argument, it seems like GCS buckets should be supported somewhere along the line.
Contributor guide
Assessment
This issue has not been assessed yet.