ARBML / ARBML/tkseem

using tk.sentencepeicetokenizer for my trained tokeinzer

Open
#10 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
113
Forks
22
PR merge metrics
No merged PRs in 30d

Description

Hello, HOw can I use a toeknizer that I have trained which is in .json file to be used using your sentencepeicetoeknizer to be able to do the evulation that you have considered ? Please I need help in this

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue mentions a trained tokenizer in a .json file, SentencePiece tokenization, and evaluation, but names no files, tests, or entry points. First inspect the repository's tokenizer and evaluation usage, then clarify the expected integration and define completion as successfully evaluating with the trained tokenizer.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.