google / google/langextract

extract(temperature=...) is silently ignored when config= or model= is passed

Open
#529 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
38.6k
Forks
2.7k
Avg merge
3d 15h
Merged PRs (30d)
3

Description

## Summary

`lx.extract()` accepts a top-level `temperature` argument documented as "Set to
0.0 for deterministic output". That value is only ever merged into the provider
kwargs inside the `model_id` branch of `extraction.py`. When the caller supplies
a pre-built `config=` (a `factory.ModelConfig`) or `model=` (a language model
instance), `temperature` is dropped: no merge, no error, no warning.

For the OpenAI provider the consequence is visible on the wire. The provider is
constructed with `temperature=None`, and `openai.py` only sets the parameter
when it is not None:

```python
temp = normalized_config.get('temperature', self.temperature)
if temp is not None:
api_params['temperature'] = temp
```

so the request omits `temperature` entirely and generation runs at whatever the
server default is.

## Versions affected

- Reproduced at HEAD `9853e4466c2a9c52c10280e0c6474d1c3b36dea1` (version 1.6.0).
- Also present in the released 1.x line installed from PyPI.
- Location: `langextract/extraction.py`, the `if model:` / `elif config:` /
`else:` model-resolution block. Only the final `else:` branch builds
`base_lm_kwargs` containing `"temperature": temperature`.

## Repro

Self-contained, no API key and no network. A stub replaces `openai.OpenAI` and
records the kwargs the provider passes to `chat.completions.create`, which is
the wire payload.

```python
import json, types
from unittest import mock

import langextract as lx
from langextract import factory
from langextract.core import data

CAPTURED = []

class _FakeCompletions:
def create(self, **api_params):
CAPTURED.append(api_params)
payload = json.dumps({"extractions": [{"entity": "Alice"}]})
message = types.SimpleNamespace(content=payload)
return types.SimpleNamespace(choices=[types.SimpleNamespace(message=message)])

class _FakeOpenAI:
def __init__(self, *a, **kw):
self.chat = types.SimpleNamespace(completions=_FakeCompletions())

EXAMPLES = [
data.ExampleData(
text="Alice went to Berlin.",
extractions=[data.Extraction(extraction_class="entity",
extraction_text="Alice")],
)
]

def run(label, **kw):
CAPTURED.clear()
with mock.patch("openai.OpenAI", _FakeOpenAI):
lx.extract(text_or_documents="Bob went to Paris.",
prompt_description="Extract entities.",
examples=EXAMPLES, use_schema_constraints=False, **kw)
print(label, "->", sorted(CAPTURED[0]))

run("A. model_id= + temperature=0.0",
model_id="gpt-4o-mini", api_key="not-a-real-key", temperature=0.0)

run("B. config= + temperature=0.0",
config=factory.ModelConfig(model_id="gpt-4o-mini",
provider="OpenAILanguageModel",
provider_kwargs={"api_key": "not-a-real-key"}),
temperature=0.0)
```

### Actual output (HEAD 9853e44)

```
A. model_id= + temperature=0.0 -> ['messages', 'model', 'n', 'response_format', 'temperature']
B. config= + temperature=0.0 -> ['messages', 'model', 'n', 'response_format']
```

Case B has no `temperature` key at all.

### Expected

Both calls send `temperature=0.0`. At minimum, case B should tell the caller
that the argument is being discarded rather than accepting it silently.

The workaround that does work today is putting the value inside the config:

```python
factory.ModelConfig(model_id="gpt-4o-mini",
provider_kwargs={"api_key": ..., "temperature": 0.0})
```

but nothing in the `extract()` signature or docstring suggests the top-level
argument stops working once `config=` is supplied.

## Impact

This fails silently and in the direction that is hardest to notice: the run
still succeeds and still returns plausible extractions, they are just sampled at
the server default temperature instead of the requested one.

We hit this chasing a reproducibility discrepancy against a batch-invariant vLLM
deployment. Raw API probes at temperature 0 returned byte-identical completions
across runs, but the same prompts driven through `langextract` with a `config=`
produced different extractions run to run. The missing wire parameter was the
whole cause. Because `temperature` is the standard lever for determinism, anyone
using `config=` or `model=` for evaluation harnesses, response caching, or
reproducibility claims is affected without any signal that their setting never
took effect.

`config=` is also the documented route for custom and OpenAI-compatible
providers, so the paths most likely to be used for self-hosted deterministic
setups are exactly the ones where the parameter is dropped.

## Suggested fix

When `config=` is given, merge an explicit `temperature` into
`config.provider_kwargs`, with the explicit argument taking precedence over any
value already in the config, and without mutating the caller's `ModelConfig`.

When `model=` is given the instance is already constructed, so there is no
provider-agnostic way to apply the value; a `UserWarning` there matches the
existing treatment of `use_schema_constraints` in the same branch.

Happy to send a PR; a patch with tests is ready.

Contributor guide

Open the contributing guide

Research direction

Start in langextract/extraction.py at the model-resolution block, then read the temperature handling in openai.py. Run the supplied no-network stub reproduction for model_id= and config= paths. Done means the requested temperature is reflected in the captured request for config= without mutating the caller's config, and model= receives the documented warning behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.