GoogleCloudPlatform / GoogleCloudPlatform/generative-ai
[Bug]: Minor bug in gemini batch prediction tutorial (trailing slash in gcs_uri breaks fsspec.glob pattern)
- Dominant language
- Jupyter Notebook
- Stars
- 17.7k
- Forks
- 4.5k
- Avg merge
- 12h 38m
- Merged PRs (30d)
- 42
Description
### File Name
`gemini/batch-prediction/intro_batch_prediction.ipynb`
### What happened?
In the current `gemini/batch-prediction/intro_batch_prediction.ipynb` notebook, the `gcs_batch_job.dest.gcs_uri` attribute from a completed job includes a trailing slash. When this URI is used with the notebook's f-string `glob` pattern, the resulting double slash breaks the path search, causing `fsspec.glob()` to fail to retrieve the `predictions.jsonl` file.
Could this issue be resolved by using `.rstrip('/')` on the URI before concatenation?
### Relevant log output
```shell
fs = fsspec.filesystem("gcs")
# Since this pattern includes a double slash, file_paths will be an empty list.
file_paths = fs.glob(f"{gcs_batch_job.dest.gcs_uri}/*/predictions.jsonl")
if gcs_batch_job.state == "JOB_STATE_SUCCEEDED":
# Load the JSONL file into a DataFrame
df = pd.read_json(f"gs://{file_paths[0]}", lines=True)
df = df.join(pd.json_normalize(df["response"], "candidates"))
display(df)
```
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Contributor guide
Assessment
This issue has not been assessed yet.