microsoft / microsoft/WindowsAppSDK

Phi Silica ignoring stop token

Open
#5,203 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area-WCR
Dominant language
C++
Stars
4.7k
Forks
471
Avg merge
3d 13h
Merged PRs (30d)
28

Description

### Describe the bug

I see cases where the model would return a result that includes the string ..."<|end|><|user|>"...
Looks like its breaking out of the <|assistant|> token instructions.

It would be good if next version would allow
1. Better support of specifying stop token
2. Support for canceling a generation

#2 would allow to implement our own stop sequence in a streaming response.

### Steps to reproduce the bug

This took running the model at scale to detect

### Expected behavior

_No response_

### Screenshots

_No response_

### NuGet package version

None

### Packaging type

_No response_

### Windows version

_No response_

### IDE

_No response_

### Additional context

_No response_

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Phi Silica generation behavior described in issue #5203, ideally at scale, and trace the streaming generation entry point that handles stop tokens. Done means the model no longer returns the reported token sequence and the requested stop-token and generation-cancellation behavior is supported, with tests covering both.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.