GoogleCloudPlatform / GoogleCloudPlatform/generative-ai

[Bug]: Minor bug in gemini batch prediction tutorial (trailing slash in gcs_uri breaks fsspec.glob pattern)

Open
#2,179 1 comment 0 reactions 1 assignee Claimed by @holtskinner View on GitHub
Dominant language
Jupyter Notebook
Stars
17.7k
Forks
4.5k
Avg merge
12h 38m
Merged PRs (30d)
42

Description

### File Name

`gemini/batch-prediction/intro_batch_prediction.ipynb`

### What happened?

In the current `gemini/batch-prediction/intro_batch_prediction.ipynb` notebook, the `gcs_batch_job.dest.gcs_uri` attribute from a completed job includes a trailing slash. When this URI is used with the notebook's f-string `glob` pattern, the resulting double slash breaks the path search, causing `fsspec.glob()` to fail to retrieve the `predictions.jsonl` file.

Could this issue be resolved by using `.rstrip('/')` on the URI before concatenation?

### Relevant log output

```shell
fs = fsspec.filesystem("gcs")

# Since this pattern includes a double slash, file_paths will be an empty list.
file_paths = fs.glob(f"{gcs_batch_job.dest.gcs_uri}/*/predictions.jsonl")

if gcs_batch_job.state == "JOB_STATE_SUCCEEDED":
# Load the JSONL file into a DataFrame
df = pd.read_json(f"gs://{file_paths[0]}", lines=True)

df = df.join(pd.json_normalize(df["response"], "candidates"))
display(df)
```

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.