microsoft / microsoft/WindowsAppSDK
Phi Silica ignoring stop token
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 471
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 28
Description
### Describe the bug
I see cases where the model would return a result that includes the string ..."<|end|><|user|>"...
Looks like its breaking out of the <|assistant|> token instructions.
It would be good if next version would allow
1. Better support of specifying stop token
2. Support for canceling a generation
#2 would allow to implement our own stop sequence in a streaming response.
### Steps to reproduce the bug
This took running the model at scale to detect
### Expected behavior
_No response_
### Screenshots
_No response_
### NuGet package version
None
### Packaging type
_No response_
### Windows version
_No response_
### IDE
_No response_
### Additional context
_No response_
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Phi Silica generation behavior described in issue #5203, ideally at scale, and trace the streaming generation entry point that handles stop tokens. Done means the model no longer returns the reported token sequence and the requested stop-token and generation-cancellation behavior is supported, with tests covering both.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- desktop
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100