libredb / libredb/libredb-studio

temperature and top_p sent together break every current Claude model via the custom provider

Open
#874 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
726
Forks
119
Avg merge
7h 41m
Merged PRs (30d)
284

Description

## Summary

When configuring the built-in AI investigation agent with `LLM_PROVIDER=custom` pointed at Anthropic's OpenAI-compatible endpoint (`https://api.anthropic.com/v1`), every agent run fails immediately. The UI shows a generic error:

> The model provider is not configured or could not be reached.

The actual server-side error (visible in container logs) is:

```
LLMStreamError: `temperature` and `top_p` cannot both be specified for this model. Please use only one.
```

(For `claude-sonnet-5` specifically, `temperature` alone is already rejected: `` `temperature` is deprecated for this model. ``)

## Root cause

The agent's default sampling settings (bundled `model-tuning` config, `defaults.sampling` = `{temperature: 0, topP: 1}`) are applied per model via `samplingFor()`, and the zod schema backing both the bundled tuning table **and** the operator-supplied `AGENT_MODEL_TUNING_PATH` override requires both fields as non-optional numbers:

```ts
strictObject({
temperature: z.number().min(0).max(2),
topP: z.number().min(0).max(1),
})
```

There's no way, via `AGENT_MODEL_TUNING_PATH` or any other configuration, to send only one of the two — both are always forwarded as `temperature` + `top_p` in the OpenAI-compatible request body.

Anthropic's API (tested on `claude-sonnet-5`, `claude-sonnet-4-6`, `claude-haiku-4-5` — current-generation models) rejects any request specifying both parameters simultaneously:

```json
{"error":{"code":"invalid_request_error","message":"`temperature` and `top_p` cannot both be specified for this model. Please use only one.","type":"invalid_request_error","param":null}}
```

`claude-sonnet-5` additionally rejects `temperature` outright even alone (it's an adaptive-thinking model; sampling params were removed for it).

## Steps to reproduce

1. Get an Anthropic API key from console.anthropic.com.
2. Set:
```
LLM_PROVIDER=custom
LLM_API_URL=https://api.anthropic.com/v1
LLM_MODEL=claude-haiku-4-5
LLM_API_KEY=
```
3. Start an AI investigation against any connected database.
4. UI shows: "The model provider is not configured or could not be reached."
5. Server log shows the real cause (see Summary above).

## Confirmed independent of app version

Compared the compiled `model-tuning` and agent request-construction chunks between `latest` (0.15.0) and `dev` (0.16.0) — byte-identical output (same webpack content hash for those modules), so this affects both.

## Environment

- Image: `ghcr.io/libredb/libredb-studio:latest` (0.15.0) and `:dev` (0.16.0)
- Confirmed via direct calls to `api.anthropic.com` (bypassing the app) that: the API itself works fine when only one of the two params is sent, and that the plain non-tool-calling `custom` provider path (which only ever sends `temperature`) works correctly. The bug is specific to the tool-calling agent path, which always sends both.

## Suggested fix

Either:
- Make `topP` optional in the model-tuning schema (send `top_p` only when explicitly configured for a model), defaulting to omitting it, or
- Add provider/endpoint-specific request shaping — skip `top_p` (or `temperature`) when the target API returns this specific 400, or when `LLM_API_URL` targets a known Anthropic-compatible endpoint.

Contributor guide

Open the contributing guide

Research direction

Start with the model-tuning configuration and the samplingFor() path described in the issue, then trace how the zod schema and AGENT_MODEL_TUNING_PATH values become the tool-calling request body. Done means the custom Anthropic-compatible agent no longer sends incompatible sampling parameters together, while supported model configurations continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, backend-api-design
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
67/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.