microsoft / microsoft/onnxruntime-inference-examples
android app token to string issue
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 414
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 14
Description
Hello. I am trying to run phi-3.5 ONNX on Android. I'm reaching out because I'm not sure how to resolve the issue related to token to string conversion. This occurs when I instruct it to output in a different language, and I confirm through the logs that the output is not being generated.
2024-10-23 17:06:06.369 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Generated token: 31734: 안
2024-10-23 17:06:06.369 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Time taken to generate token: 1.32 seconds
2024-10-23 17:06:06.433 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Generated token: 238:
2024-10-23 17:06:06.433 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Time taken to generate token: 0.064 seconds
2024-10-23 17:06:06.495 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Generated token: 136:
2024-10-23 17:06:06.495 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Time taken to generate token: 0.062 seconds
2024-10-23 17:06:06.556 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Generated token: 152:
2024-10-23 17:06:06.556 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Time taken to generate token: 0.061 seconds
2024-10-23 17:06:06.618 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Generated token: 30944: 하
2024-10-23 17:06:06.618 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Time taken to generate token: 0.062 seconds
2024-10-23 17:06:06.679 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Generated token: 31578: 세
2024-10-23 17:06:06.679 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Time taken to generate token: 0.061 seconds
2024-10-23 17:06:06.682 29366-29366 time.genai.demo ai.onnxruntime.genai.demo I Waiting for a blocking GC NativeAlloc
2024-10-23 17:06:06.692 29366-29366 time.genai.demo ai.onnxruntime.genai.demo I WaitForGcToComplete blocked NativeAlloc on NativeAlloc for 10.871ms
2024-10-23 17:06:06.695 29366-29378 InputTransport ai.onnxruntime.genai.demo D Input channel destroyed: 'ClientS', fd=135
2024-10-23 17:06:06.759 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Generated token: 31527: 요
2024-10-23 17:06:06.759 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Time taken to generate token: 0.08 seconds
2024-10-23 17:06:06.820 29366-29512 genai.demo.MainActivity ai.onnxruntime.genai.demo I Generated token: 29991: !
When I execute the phi 3.5 tokenizer in Python, the output is 안녕하세요!, but the android output is 안하세요!.
I want to decode three tokens([238, 136, 152]) to obtain the correct results. I would appreciate any guidance on how to achieve this.
Thank you.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the Android genai.demo.MainActivity token-generation and logging path, then compare its decoding of tokens [238, 136, 152] with the phi-3.5 tokenizer output in Python. Done means the Android output includes 안녕하세요! rather than 안하세요!.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, python
- Domain
- machine-learning, mobile-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100