extract(temperature=...) is silently ignored when config= or model= is passed
- Dominant language
- Python
- Stars
- 38.6k
- Forks
- 2.7k
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 3
Description
## Summary
`lx.extract()` accepts a top-level `temperature` argument documented as "Set to
0.0 for deterministic output". That value is only ever merged into the provider
kwargs inside the `model_id` branch of `extraction.py`. When the caller supplies
a pre-built `config=` (a `factory.ModelConfig`) or `model=` (a language model
instance), `temperature` is dropped: no merge, no error, no warning.
For the OpenAI provider the consequence is visible on the wire. The provider is
constructed with `temperature=None`, and `openai.py` only sets the parameter
when it is not None:
```python
temp = normalized_config.get('temperature', self.temperature)
if temp is not None:
api_params['temperature'] = temp
```
so the request omits `temperature` entirely and generation runs at whatever the
server default is.
## Versions affected
- Reproduced at HEAD `9853e4466c2a9c52c10280e0c6474d1c3b36dea1` (version 1.6.0).
- Also present in the released 1.x line installed from PyPI.
- Location: `langextract/extraction.py`, the `if model:` / `elif config:` /
`else:` model-resolution block. Only the final `else:` branch builds
`base_lm_kwargs` containing `"temperature": temperature`.
## Repro
Self-contained, no API key and no network. A stub replaces `openai.OpenAI` and
records the kwargs the provider passes to `chat.completions.create`, which is
the wire payload.
```python
import json, types
from unittest import mock
import langextract as lx
from langextract import factory
from langextract.core import data
CAPTURED = []
class _FakeCompletions:
def create(self, **api_params):
CAPTURED.append(api_params)
payload = json.dumps({"extractions": [{"entity": "Alice"}]})
message = types.SimpleNamespace(content=payload)
return types.SimpleNamespace(choices=[types.SimpleNamespace(message=message)])
class _FakeOpenAI:
def __init__(self, *a, **kw):
self.chat = types.SimpleNamespace(completions=_FakeCompletions())
EXAMPLES = [
data.ExampleData(
text="Alice went to Berlin.",
extractions=[data.Extraction(extraction_class="entity",
extraction_text="Alice")],
)
]
def run(label, **kw):
CAPTURED.clear()
with mock.patch("openai.OpenAI", _FakeOpenAI):
lx.extract(text_or_documents="Bob went to Paris.",
prompt_description="Extract entities.",
examples=EXAMPLES, use_schema_constraints=False, **kw)
print(label, "->", sorted(CAPTURED[0]))
run("A. model_id= + temperature=0.0",
model_id="gpt-4o-mini", api_key="not-a-real-key", temperature=0.0)
run("B. config= + temperature=0.0",
config=factory.ModelConfig(model_id="gpt-4o-mini",
provider="OpenAILanguageModel",
provider_kwargs={"api_key": "not-a-real-key"}),
temperature=0.0)
```
### Actual output (HEAD 9853e44)
```
A. model_id= + temperature=0.0 -> ['messages', 'model', 'n', 'response_format', 'temperature']
B. config= + temperature=0.0 -> ['messages', 'model', 'n', 'response_format']
```
Case B has no `temperature` key at all.
### Expected
Both calls send `temperature=0.0`. At minimum, case B should tell the caller
that the argument is being discarded rather than accepting it silently.
The workaround that does work today is putting the value inside the config:
```python
factory.ModelConfig(model_id="gpt-4o-mini",
provider_kwargs={"api_key": ..., "temperature": 0.0})
```
but nothing in the `extract()` signature or docstring suggests the top-level
argument stops working once `config=` is supplied.
## Impact
This fails silently and in the direction that is hardest to notice: the run
still succeeds and still returns plausible extractions, they are just sampled at
the server default temperature instead of the requested one.
We hit this chasing a reproducibility discrepancy against a batch-invariant vLLM
deployment. Raw API probes at temperature 0 returned byte-identical completions
across runs, but the same prompts driven through `langextract` with a `config=`
produced different extractions run to run. The missing wire parameter was the
whole cause. Because `temperature` is the standard lever for determinism, anyone
using `config=` or `model=` for evaluation harnesses, response caching, or
reproducibility claims is affected without any signal that their setting never
took effect.
`config=` is also the documented route for custom and OpenAI-compatible
providers, so the paths most likely to be used for self-hosted deterministic
setups are exactly the ones where the parameter is dropped.
## Suggested fix
When `config=` is given, merge an explicit `temperature` into
`config.provider_kwargs`, with the explicit argument taking precedence over any
value already in the config, and without mutating the caller's `ModelConfig`.
When `model=` is given the instance is already constructed, so there is no
provider-agnostic way to apply the value; a `UserWarning` there matches the
existing treatment of `use_schema_constraints` in the same branch.
Happy to send a PR; a patch with tests is ready.
Contributor guide
Research direction
Start in langextract/extraction.py at the model-resolution block, then read the temperature handling in openai.py. Run the supplied no-network stub reproduction for model_id= and config= paths. Done means the requested temperature is reflected in the captured request for config= without mutating the caller's config, and model= receives the documented warning behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100