apache / apache/texera

Malformed chat responses abort the run in the text-generation and QA codegens

Open
#8,486 1 comment 0 reactions 1 assignee Claimed by @PG1204 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### What happened?

`TextGenCodegen.parsePython` extracts the chat response with no guard at all:

return body["choices"][0]["message"]["content"]

`QaRankingCodegen.parsePython` guards only on key presence, at three sites:

if "choices" in body:
return body["choices"][0]["message"]["content"]

Both raise on a malformed provider response rather than degrading:

- `{"choices": []}` → `IndexError` (the `in body` check passes, the list is empty)
- `{"choices": [{}]}` → `KeyError: 'message'`
- `{"choices": [{"message": {}}]}` → `KeyError: 'content'`

`parsePython` runs once per row, so a single malformed response from one provider aborts the entire run. The native `hf-inference` paths in the same functions already degrade politely via `body.get("answer", json.dumps(body))`, so the two paths are inconsistent.

This path is reachable in practice: of the top 100 models by downloads, 45 for `text-generation`, 37 for `question-answering` and 72 for `sentence-similarity` have a live third-party provider, so the chat branch does execute for these tasks.

This is the same defect #7920 fixed in `ImageTaskCodegen`, raised in review by @Ma77Ball. The remaining sites are in code merged earlier (#7798) and were left out of that PR to keep the diff focused.

### How to reproduce?

Run the operator with `text-generation` (or any of the QA/ranking tasks) against a model whose cheapest provider is a third-party chat provider, and have that provider return a 200 with an empty `choices` list or a choice missing `message`/`content`. The operator raises and the run aborts, instead of writing the raw body into the result column.

### Version/Branch

1.4.0-incubating-SNAPSHOT (main)

### Commit Hash (Optional)

_No response_

### What browsers are you seeing the problem on?

_No response_

### Relevant log output

```shell

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.