dair-ai / dair-ai/Prompt-Engineering-Guide

Feedback for “LLM Settings”

Open
#263 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
78.3k
Forks
8.6k
PR merge metrics
No merged PRs in 30d

Description

> The general recommendation is to alter one, not both.

I noticed in the explanation that the general recommendation is to alter either `temperature` or `top_p`, but not both. However, to me, it seems to make sense to use both:

- If `temperature` is set to 0, it would seem that `top_p` has no effect, as the output will always be deterministic, picking the most likely next token.
- Conversely, if `temperature` is set to 1, allowing for more creativity, but `top_p` is set to a low value like 0.01, it seems that the creativity could be effectively restricted to only the top 1% of probable words.

Wouldn't it then be practical to use both parameters together? `Temperature` could control the overall creativity of the model, while `top_p` sets a boundary on the word options for the next token. This could allow for a more nuanced control of the model's output, fitting various use cases. Could you clarify if this approach is indeed viable, or if there are potential drawbacks or complexities I'm overlooking?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.