posit-dev / posit-dev/chatlas

OpenAI provider incorrectly maps params(top_k=) to top_logprobs

Open
#412 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ai-triage:done Priority: Medium
Dominant language
Python
Stars
176
Forks
28
Avg merge
18h 42m
Merged PRs (30d)
16

Description

Summary

chatlas/_provider_openai.py maps the standardized top_k chat parameter onto OpenAI's top_logprobs field:

# chatlas/_provider_openai.py:596-597
if "top_k" in params:
    res["top_logprobs"] = params["top_k"]

These are unrelated parameters — top_k controls sampling (limiting the candidate token pool), while top_logprobs controls how many log-probabilities are returned per token. Conflating them means setting top_k silently changes response metadata (logprobs count) instead of doing anything related to sampling, and vice versa.

ellmer just fixed the equivalent bug across its OpenAI-based providers (tidyverse/ellmer#1115, tidyverse/ellmer#1113): top_k is no longer sent as top_logprobs for chat_openai(), chat_deepseek(), or the generic OpenAI-compatible provider.

Expected fix

Stop mapping top_ktop_logprobs in chatlas/_provider_openai.py (and check chatlas/_provider_openai_completions.py / other OpenAI-compatible providers for the same pattern). Since OpenAI's API doesn't support top_k for sampling, the standardized top_k param should likely be dropped/warned-on for OpenAI-based providers rather than silently repurposed, matching ellmer's resolution.

References

  • ellmer PR: tidyverse/ellmer#1115 (issue: tidyverse/ellmer#1113)
  • chatlas: chatlas/_provider_openai.py:596-597

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at chatlas/_provider_openai.py:596-597 and inspect how standardized parameters are translated. Check chatlas/_provider_openai_completions.py and other OpenAI-compatible providers for the same mapping. Done means top_k is not silently mapped to top_logprobs, with unsupported handling matching the expected provider behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.