WeblateOrg / WeblateOrg/weblate

`autotranslate` reports machinery failures as success, with no way to distinguish "nothing to do" from "engine errored"

Open
#20,996 0 comments 1 reaction 0 assignees View on GitHub
Area: API Area: Automatic translation Waiting for: Demand
Dominant language
Python
Stars
6.1k
Forks
1.4k
Avg merge
9h 53m
Merged PRs (30d)
395

Description

### Describe the issue

When a machine translation engine raises `MachineTranslationError` during `autotranslate`, the exception is caught and logged, but nothing about the failure is surfaced to the API caller. The response is indistinguishable from the case where there was genuinely nothing left to translate:

```
POST /api/translations/{p}/{c}/{lang}/autotranslate/
→ 200 {"details": "Automatic translation completed, no strings were updated."}
```

This showed up independently of the `persona`/`style` `KeyError` I filed in #20993 — in one test run, a correctly configured engine hit a parsing failure on its provider's reply:

```
machinery[Mistral]: Mismatching assistant reply.: MachineTranslationError
//: failed automatic translation: Mismatching assistant reply.
```

Every string in that batch was silently dropped, while a sibling language in the same run translated successfully — so the engine and configuration were both fine; the provider just failed to return a parseable reply for that batch. The API caller has no way to detect this short of tailing server logs. This makes any MT provider outage or transient failure invisible to automation built on top of `autotranslate`.

I believe this overlaps significantly with #18613 ("Machinery config split: batch translation bypasses project-level settings with no UI indication"), which already raised that machinery failures/skips aren't surfaced to the caller. The fix merged for that issue (#18618) addressed the project-vs-site-level settings display split specifically, but didn't change error propagation — the underlying "failures are invisible to the caller" gap it identified still exists for this different trigger.

### I already tried
- [x] I've read and searched the documentation.
- [x] I've searched for similar filed issues in this repository. (See #18613 above — related but not a full duplicate of this specific gap.)

### Steps to reproduce the behavior

1. Configure a machinery service such that a call will raise `MachineTranslationError` for at least one batch (e.g. a provider reply that fails Weblate's own response parsing).
2. Run `autotranslate` against strings that should match.
3. Observe `200 {"details": "Automatic translation completed, no strings were updated."}` — a success-shaped response — while the server log shows the actual `MachineTranslationError`.

### Expected behavior

Some way for `autotranslate` to distinguish a real failure from "nothing needed updating" — e.g. a partial-failure count, an `errors` array listing which engines/batches failed and why, or a non-2xx status when every configured engine failed outright. Partial success should remain possible (some engines/batches can legitimately succeed while others fail) rather than the whole request needing to fail.

### Screenshots

N/A

### Exception traceback

```
machinery[Mistral]: Mismatching assistant reply.: MachineTranslationError
//: failed automatic translation: Mismatching assistant reply.
```

### How do you run Weblate?

Docker container

### Weblate version

2026.7.1 (confirmed against the current `latest` Docker tag as well)

### Weblate deploy checks

N/A — reproduced on a minimal, disposable single-node instance for isolation.

### Additional context

Related: #18613 / #18618, and #20993 (a different trigger for the same "failures are invisible to the caller" gap).

Contributor guide

Open the contributing guide

Research direction

Start at the POST /api/translations/{p}/{c}/{lang}/autotranslate/ entry point and trace where MachineTranslationError is caught and logged. Compare the genuine no-op response with a provider parsing failure, including partial success and all-engines-failed cases. Done means API callers can distinguish these outcomes instead of receiving a success-shaped no-op response for failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, localization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.