Plan agent connectivity to the backend API from outside the cluster

Still undecided, and the option space below still stands. Two things have changed since this was
written: a default has emerged by accident, and that default is currently broken for the exact
workload it needs to carry.

## Current state (2026-09)

Staging and production now have an `ingress.yaml`, but it defines **only** the frontend route:

```yaml
# The SPA pod's nginx reverse-proxies /api/ to smartem-http-api-service
# internally, so a single route covering / is sufficient.
- host: smartem.diamond.ac.uk
http:
paths:
- path: /
backend:
service:
name: smartem-frontend-service
```

There is no backend ingress. However, an agent *could* reach the backend at
`smartem.diamond.ac.uk/api/...`, because the SPA pod's nginx proxies `/api/` onward. In other
words **Option B(i) is now the de facto answer**, arrived at as a side effect of the frontend
deployment rather than by decision - and B(i) is the option this issue argues hardest against,
on the grounds that it couples machine traffic to a pod sized for browser navigations.

## The current route does not support SSE

The `/api/` block in `apps/smartem/nginx.conf` (smartem-frontend) contains no streaming
configuration:

```nginx
location /api/ {
set $backend_upstream "${BACKEND_HOST}";
rewrite ^/api/(.*) /$1 break;
proxy_pass http://$backend_upstream;
proxy_set_header Host $host;
...
}
```

There is no `proxy_buffering off`, no extended `proxy_read_timeout`, and no
`proxy_http_version 1.1`. Under nginx defaults that means responses are **buffered**, so events
do not stream as they are produced, and the read timeout is **60 seconds**, so any stream idle
for a minute is terminated.

Since long-lived SSE is the primary reason the agent needs this route at all, the accidental
default fails at precisely the thing it exists for.

**This affects the front end too, not only the agent.** Frontend SSE consumption
(smartem-frontend#75) will meet the same configuration. It will appear to work in local
development, where Vite proxies directly to the backend and never traverses this nginx, and then
fail in staging and production. That is a latent defect waiting for the next piece of frontend
work rather than a hypothetical.

Note also that ingress controllers apply their own buffering and timeout defaults, so fixing the
pod's nginx is necessary but may not be sufficient - whichever route is chosen needs
SSE-appropriate settings at every hop.

## What this changes about the decision

The choice is now between **ratifying B(i) and configuring it properly**, or **moving to a
separate backend route (Option A)** and leaving the SPA proxy for browser traffic only.

Arguments that have strengthened since the original write-up:

- The SSE defect is real and must be fixed under *any* option, because the front end needs
streaming through whichever path it uses. It is therefore not a differentiator between the
options, but it does mean "do nothing" is no longer a neutral position.
- Separating machine traffic from browser traffic (Option A) means SSE tuning, timeouts and
sizing can be set for streaming without compromising the SPA's settings, and the two failure
domains stay independent. That was already the argument for A; the discovery that the shared
path needs materially different tuning makes it more concrete.

## Related, updated

- smartem-devtools#181 - **closed as completed**. The proxy-pass architecture it introduced is
what created the accidental default described above.
- smartem-devtools#179 - **closed as superseded** by the wider deployment rework. This issue is
now where the connectivity question actually survives.
- smartem-devtools#205 - closed; added the frontend ingress and explicitly deferred this.
- smartem-frontend#75 - frontend SSE consumption; will be affected by whatever is decided here,
and by the buffering defect regardless.

## Out of scope

Implementing the choice, as before. Note however that the nginx SSE configuration is a defect
rather than part of this decision, and does not need to wait for it.

Lies die aktuellen staging- und production-ingress.yaml sowie apps/smartem/nginx.conf und vergleiche Option A mit B(i), einschließlich des SSE-Pufferungs- und Timeout-Verhaltens an jedem Hop. Prüfe die verknüpften frontend- und deployment-Issues; als abgeschlossen gilt dies, wenn die Konnektivitätsoption und ihre SSE-Anforderungen ausdrücklich entschieden und dokumentiert sind, nicht wenn sie hier implementiert wurden.

Vom Indexierungsmodell aus dem Issue-Text verfasst.