Batch path nests tools inside generationConfig, producing an invalid batch request
- Dominant language
- Python
- Stars
- 38.6k
- Forks
- 2.7k
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 3
Description
**Environment:** langextract at HEAD 9853e44, Vertex AI backend, batch mode.
**Bug:** `GeminiLanguageModel` with `tools` configured produces an invalid batch request. The batch branch in `langextract/providers/gemini.py` pops `system_instruction` and `safety_settings` out of the config to place them at the request top level, but never pops `tools` (allowlisted in `_API_CONFIG_KEYS` at gemini.py:112 and documented in the constructor as forwarded to the API). `gemini_batch._build_request` then camelizes the leftover key into `generationConfig`, producing:
```json
{"generationConfig": {"temperature": 0.0, "tools": [{"google_search": {}}], "candidateCount": 1}}
```
In the `GenerateContentRequest` schema, `tools` is a top-level request field like `systemInstruction` and `safetySettings`, not a `GenerationConfig` field, so Vertex batch validation rejects the request. With `ignore_item_errors` the outputs silently pad to empty strings instead. The identical model config works on the realtime path because `generate_content(config=call_config)` accepts tools at that level.
**Repro (no API call needed):**
```python
from langextract.providers import gemini_batch
gen_config = {"temperature": 0.0, "tools": [{"google_search": {}}], "candidate_count": 1}
req = gemini_batch._build_request(prompt="p", gen_config=gen_config, ...)
# req["generationConfig"] contains "tools"; the schema wants it at req top level
```
**Trigger:** `GeminiLanguageModel(vertexai=True, project=..., location=..., tools=[{"google_search": {}}], batch={"threshold": 1})` then extract/infer over >= threshold prompts.
**Fix shape:** pop `tools` in the batch branch next to the `system_instruction`/`safety_settings` pops and pass it through `infer_batch` into `_build_request`, which sets `request["tools"]` at the top level, mirroring the existing `systemInstruction`/`safetySettings` handling. Every other `_API_CONFIG_KEYS` member left in the dict is a legitimate `generationConfig` field; `tools` is the single misplaced one.
I have this fix implemented with a regression test and can open a PR.
Contributor guide
Research direction
Start in langextract/providers/gemini.py at the batch branch that handles system_instruction and safety_settings, then follow infer_batch into langextract/providers/gemini_batch.py::_build_request. Use the no-API-call reproduction and the reported regression test; done means tools is emitted at the request top level rather than inside generationConfig, while the realtime path remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100