allenai / allenai/bilm-tf

Which tokenizer used for Japanese pretrained model?

Open
#210 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.6k
Forks
447
PR merge metrics
No merged PRs in 30d

Description

Hi,
I have a question on your provided pretrained model for Japanese.
Which tokenizer was used to train the model? I found the model is provided by `ExaWizards Inc. Enkhbold Bataa, Joshua Wu.`

Thanks.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.