huggingface / huggingface/candle
[feature] Add min_p sampling and frequency/presence penalties to generation
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
### Motivation
`candle_transformers::generation::Sampling` offers ArgMax, All, TopK, TopP,
TopKThenTopP and GumbelSoftmax, but not **min_p** (a popular, robust alternative
to top_p) nor OpenAI-style **frequency_penalty / presence_penalty**. These are
expected by anyone exposing an OpenAI-compatible API on top of candle.
### Gap in candle
- No `min_p` variant in `Sampling`.
- No frequency/presence penalty helpers in the generation module (repeat-penalty,
where present, lives outside this module and differs from OpenAI semantics).
### Proposed implementation
- Add `Sampling::MinP { p: f64, temperature: f64 }` (keep min-p tokens with prob
≥ `p * max_prob`), wired through `LogitsProcessor`.
- Add penalty helpers applied to logits before sampling, parameterized over the
generated-token history, matching OpenAI's frequency/presence semantics.
### Use-case / example
```rust
// min_p sampling for an OpenAI-compatible server.
let mut lp = LogitsProcessor::from_sampling(seed, Sampling::MinP { p: 0.05, temperature: 0.8 });
let next = lp.sample(&logits)?;
// frequency / presence penalties over the running output.
let logits = apply_frequency_presence_penalty(&logits, &generated, freq: 0.5, presence: 0.3)?;
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with candle_transformers::generation::Sampling and LogitsProcessor, tracing how existing sampling variants are wired. Check the generation module's tests or sampling entry points before implementing MinP and the frequency/presence penalty helpers. Done means both requested behaviors are exposed, match the stated semantics, and are covered by relevant tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100