Docsite embedding is broken (401 unauthorized)

We have a problem updating our emebddings.

We recently added an auth layer to all apollo endpoints. Requests must now include an API key, which must be registered with the apollo database. Or requests are rejected.

This is causing our production docsite to fail to load embeddings. Over in the docs repo, we call the embedding service to get it to trigger an update... and that call is getting rejected because there's no API key.

Another problem here is that the fail is silent. The GH action run passes, even though the request fails. example

Now. We should have some kind of auth on the embedding service and refuse anonymous requests. It doesn't really make sense to use an anthropic key for this. But then again the key is arbitrary, so maybe we should use the existing keygen service to generate a key for this task.

This sort of thing will apply to other services too. I do wonder if we need a different solution for basic auth.

Issues to resolve here

1. The GH action in docs should fail if the curl returns an error code
2. Add a workaround for the auth problem
3. Maybe implement a new auth strategy

Start with the docs-repository GitHub Actions run linked in the issue and inspect the curl request that triggers the embedding update. Verify how its HTTP errors are handled, then trace the embedding service authentication and key registration. Done means rejected requests fail the action and the embedding update succeeds with an approved key.