google / google/langextract

ResolverParsingError when using vLLM/lmdeploy as backend (Qwen3-32B-GPTQ)

Open
#414 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
38.6k
Forks
2.7k
Avg merge
3d 15h
Merged PRs (30d)
3

Description

When using `vllm` or `lmdeploy` as the inference backend via the OpenAI-compatible API, `langextract` fails with a `ResolverParsingError: Failed to parse JSON content: Expecting value: line 1 column 1 (char 0)`.

Interestingly, the exact same code and prompt work perfectly when using `Ollama`. It seems the JSON output from vLLM/lmdeploy is not being correctly captured or parsed by the `Resolver`, even when the model returns a valid JSON string (verified via manual `requests` calls).

```text
File ~/programs/miniconda3/envs/agents/lib/python3.12/site-packages/langextract/resolver.py:271, in Resolver.resolve(self, input_text, suppress_parse_errors, **kwargs)
267 logging.exception(
268 "Failed to parse input_text: %s, error: %s", input_text, e
269 )
270 return []
--> 271 raise ResolverParsingError(str(e)) from e

ResolverParsingError: Failed to parse JSON content: Expecting value: line 1 column 1 (char 0)
```

**Environment**
*Model*: Qwen3-32B-GPTQ-Int4
*Backend*: vLLM / lmdeploy (OpenAI-compatible server)

**Current Config**:
```text
config = lx.factory.ModelConfig(
model_id="vllm:http://localhost:8000/v1",
provider="VLLMLanguageModel",
provider_kwargs=dict(
temperature=0.7,
max_tokens=1024,
# Server connection settings
timeout=60.0,
),
)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.