nucleus sampler problem?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
In the Github repository: https://github.com/the-crypt-keeper/can-ai-code/tree/main from @the-crpyt-keeper the WizardCoder was investigated with ctranslate2.
8/08 Added cformers2 support and evaluated michaelfeil/ct2fast-WizardCoder-15B-V1.0 it seems this runtime may have a problem with it's nucleus sampler, precise settings hurt the results far more then they should.
Can this be investigated in more detail?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the linked can-ai-code WizardCoder evaluation and the referenced ctranslate2 nucleus-sampling settings. Reproduce the comparison between precise and less precise settings, then determine whether the sampler behavior is a CTranslate2 defect. Done means the behavior is explained and, if confirmed, covered by a regression test and fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100