ResolverParsingError when using vLLM/lmdeploy as backend (Qwen3-32B-GPTQ)
- Dominant language
- Python
- Stars
- 38.6k
- Forks
- 2.7k
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 3
Description
When using `vllm` or `lmdeploy` as the inference backend via the OpenAI-compatible API, `langextract` fails with a `ResolverParsingError: Failed to parse JSON content: Expecting value: line 1 column 1 (char 0)`.
Interestingly, the exact same code and prompt work perfectly when using `Ollama`. It seems the JSON output from vLLM/lmdeploy is not being correctly captured or parsed by the `Resolver`, even when the model returns a valid JSON string (verified via manual `requests` calls).
```text
File ~/programs/miniconda3/envs/agents/lib/python3.12/site-packages/langextract/resolver.py:271, in Resolver.resolve(self, input_text, suppress_parse_errors, **kwargs)
267 logging.exception(
268 "Failed to parse input_text: %s, error: %s", input_text, e
269 )
270 return []
--> 271 raise ResolverParsingError(str(e)) from e
ResolverParsingError: Failed to parse JSON content: Expecting value: line 1 column 1 (char 0)
```
**Environment**
*Model*: Qwen3-32B-GPTQ-Int4
*Backend*: vLLM / lmdeploy (OpenAI-compatible server)
**Current Config**:
```text
config = lx.factory.ModelConfig(
model_id="vllm:http://localhost:8000/v1",
provider="VLLMLanguageModel",
provider_kwargs=dict(
temperature=0.7,
max_tokens=1024,
# Server connection settings
timeout=60.0,
),
)
```
Contributor guide
Assessment
This issue has not been assessed yet.