google / google/uncertainty-baselines

Where is keras_tokenizer.json supposed to come from for clinc_intent.py?

Open
#593 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1.6k
Forks
224
Avg merge
15h 36m
Merged PRs (30d)
2

Description

Hi There,

I'm trying to do a simple smoke test of uncertainty-baselines and thought I'd use baselines/clinc_intent/deterministic.py . However, I'm not entirely sure I understand how to run this example. In particular where is keras_tokenizer.json supposed to come from?

Here's what I've tried:
pip install --upgrade pip setuptools
pip install --editable .[tensorflow,models,datasets]

tfds build --data_dir tensorflow_datasets clinc_oos

python baselines/clinc_intent/deterministic.py \
--data_dir=tensorflow_datasets/clinc_oos \
--bert_dir=models/uncased_L-12_H-768_A-12 \
--output_dir=/tmp/model \
--use_gpu=True \
--num_cores=1

Which generates this error:
File "/mnt/shared_ad2_mt1/jpeck/src/uncertainty-baselines/venv/lib64/python3.6/site-packages/tensorflow/python/lib/io/file_io.py", line 80, in _preread_check
compat.path_to_str(self.__name), 1024 * 512)
tensorflow.python.framework.errors_impl.NotFoundError: tensorflow_datasets/clinc_oos/keras_tokenizer.json; No such file or directory

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.