google-deepmind / google-deepmind/videoprism

Text Tokenizer - GCS Bucket Permission Denied

Open
#63 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
392
Forks
39
Avg merge
14h 35m
Merged PRs (30d)
1

Description

Hi, I'm working with `videoprism_lvt_public_v1_base` like so:

```
from videoprism import models as vp
vp.load_text_tokenizer(tokenizer_name)
```

However, this gives me a permission denied error as the tokenizer is now hosted in a now-private GCS bucket: gs://t5-data/vocabs/cc_en.32000/sentencepiece.model.

I've also posted an issue here: https://github.com/google/sentencepiece/issues/1191, but any other help would be much appreciated. Thanks!

Contributor guide

Open the contributing guide

Research direction

Start at `vp.load_text_tokenizer(tokenizer_name)` and investigate how it accesses `gs://t5-data/vocabs/cc_en.32000/sentencepiece.model`. Reproduce the permission-denied failure and determine whether the issue is in the repository's tokenizer loading path or access to the now-private bucket; done means the tokenizer can be loaded or the required access limitation is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.