google / google/langextract

Batch path nests tools inside generationConfig, producing an invalid batch request

Open Beginner friendly
#532 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
38.6k
Forks
2.7k
Avg merge
3d 15h
Merged PRs (30d)
3

Description

**Environment:** langextract at HEAD 9853e44, Vertex AI backend, batch mode.

**Bug:** `GeminiLanguageModel` with `tools` configured produces an invalid batch request. The batch branch in `langextract/providers/gemini.py` pops `system_instruction` and `safety_settings` out of the config to place them at the request top level, but never pops `tools` (allowlisted in `_API_CONFIG_KEYS` at gemini.py:112 and documented in the constructor as forwarded to the API). `gemini_batch._build_request` then camelizes the leftover key into `generationConfig`, producing:

```json
{"generationConfig": {"temperature": 0.0, "tools": [{"google_search": {}}], "candidateCount": 1}}
```

In the `GenerateContentRequest` schema, `tools` is a top-level request field like `systemInstruction` and `safetySettings`, not a `GenerationConfig` field, so Vertex batch validation rejects the request. With `ignore_item_errors` the outputs silently pad to empty strings instead. The identical model config works on the realtime path because `generate_content(config=call_config)` accepts tools at that level.

**Repro (no API call needed):**

```python
from langextract.providers import gemini_batch
gen_config = {"temperature": 0.0, "tools": [{"google_search": {}}], "candidate_count": 1}
req = gemini_batch._build_request(prompt="p", gen_config=gen_config, ...)
# req["generationConfig"] contains "tools"; the schema wants it at req top level
```

**Trigger:** `GeminiLanguageModel(vertexai=True, project=..., location=..., tools=[{"google_search": {}}], batch={"threshold": 1})` then extract/infer over >= threshold prompts.

**Fix shape:** pop `tools` in the batch branch next to the `system_instruction`/`safety_settings` pops and pass it through `infer_batch` into `_build_request`, which sets `request["tools"]` at the top level, mirroring the existing `systemInstruction`/`safetySettings` handling. Every other `_API_CONFIG_KEYS` member left in the dict is a legitimate `generationConfig` field; `tools` is the single misplaced one.

I have this fix implemented with a regression test and can open a PR.

Contributor guide

Open the contributing guide

Research direction

Start in langextract/providers/gemini.py at the batch branch that handles system_instruction and safety_settings, then follow infer_batch into langextract/providers/gemini_batch.py::_build_request. Use the no-API-call reproduction and the reported regression test; done means tools is emitted at the request top level rather than inside generationConfig, while the realtime path remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
api, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.