Review and propose a better solution for adaptor docs rag

This is an exploratory issue to improve RAG for adaptor docs (by which we mean adaptor API reference docs, like this https://docs.openfn.org/adaptors/packages/dhis2-docs).

The headline story here is this:

I do not trust the process right now for feeding adaptor APIs into job_chat prompts.

Our approach has changed over the years, I think right now what we do is this:

• When a query comes in, the job_chat services decides whether we need to run RAG or not
• If RAG is needed, we do a general seach on the docsite embeddings and build a list of relevent docs
• Then job_chat goes off an builds a prompt to send to Anthropic
• If relevant docs are appended to the chat session, they'll be attached to the prompt
• ENTIRELY SEPERATELY TO ALL this, if the chat session is about a job with an adaptor (ie, all the time), we load signatures for the adaptor and append them to the prompt

The problems are:

• The adaptors API reference contains main important information beyond signatures: including textual descriptions, object interfaces and examples (the examples seem particularly important)
• Sending the signtures sends a very terse set of data to the assistant. Some adaptors (particularly the FHIR ones) contain a lot of sigantures too, so it takes up a lot of context
• We tried including adaptor docs as part of general rag, but the API listings are quite terse and do not embed well

What I'm really looking for is a process where we scan the list of functions for the relevant adaptor, pick out signatures that feel like they may be relevant, and append full docs + relevant signatures to the prompt. It's possible that no functions are relevant, or all (in which case maybe examples aren't needed). This RAG phase should be intelligent and guided around the structure and nature of adaptor APIs; it should not be a general similarity search.

I want want to test this not by looking at the output from the model and deciding if it works, but by looking at the prompt and deciding if relevant docs have been attached.

Related Issues

There are a bunch of issues around this, likely old and many condadictory. Part of this story is to close down irrelevant tickets and make sure that the core concern is documented in one place.

• https://github.com/OpenFn/apollo/issues/396
• https://github.com/OpenFn/apollo/issues/211
• https://github.com/OpenFn/apollo/issues/205
• Maybe https://github.com/OpenFn/apollo/issues/503
• https://github.com/OpenFn/apollo/issues/180

No files or tests are named. Start by tracing the job_chat service's RAG decision, docsite embeddings, adaptor signature loading, and prompt assembly. Done means a documented or tested process that attaches relevant adaptor documentation and signatures, evaluated by inspecting prompts rather than model output.