Malformed chat responses abort the run in the text-generation and QA codegens
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### What happened?
`TextGenCodegen.parsePython` extracts the chat response with no guard at all:
return body["choices"][0]["message"]["content"]
`QaRankingCodegen.parsePython` guards only on key presence, at three sites:
if "choices" in body:
return body["choices"][0]["message"]["content"]
Both raise on a malformed provider response rather than degrading:
- `{"choices": []}` → `IndexError` (the `in body` check passes, the list is empty)
- `{"choices": [{}]}` → `KeyError: 'message'`
- `{"choices": [{"message": {}}]}` → `KeyError: 'content'`
`parsePython` runs once per row, so a single malformed response from one provider aborts the entire run. The native `hf-inference` paths in the same functions already degrade politely via `body.get("answer", json.dumps(body))`, so the two paths are inconsistent.
This path is reachable in practice: of the top 100 models by downloads, 45 for `text-generation`, 37 for `question-answering` and 72 for `sentence-similarity` have a live third-party provider, so the chat branch does execute for these tasks.
This is the same defect #7920 fixed in `ImageTaskCodegen`, raised in review by @Ma77Ball. The remaining sites are in code merged earlier (#7798) and were left out of that PR to keep the diff focused.
### How to reproduce?
Run the operator with `text-generation` (or any of the QA/ranking tasks) against a model whose cheapest provider is a third-party chat provider, and have that provider return a 200 with an empty `choices` list or a choice missing `message`/`content`. The operator raises and the run aborts, instead of writing the raw body into the result column.
### Version/Branch
1.4.0-incubating-SNAPSHOT (main)
### Commit Hash (Optional)
_No response_
### What browsers are you seeing the problem on?
_No response_
### Relevant log output
```shell
```
Contributor guide
Assessment
This issue has not been assessed yet.