Metadata breaks while an adaptor is installing, and when its version isn't a real one

When you open a job in the editor we fetch the adaptor so we can offer autocomplete and docs. If the adaptor isn't already installed on that web pod we install it on the spot. While that install is running, a second request for the same adaptor gets handed a half-built entry, and breaks.

This is one bug with two faces, and together they account for around 656 errors in production.

What happens

Before we start installing, we add the adaptor to our in-memory list using the version string we were asked for, marked as installing, with no path yet. Anything that looks the adaptor up during that window finds that entry.

What happens next depends on the version:

• A pinned version, e.g. @7.3.2. The entry matches, so the caller gets it back, but it has no path. The metadata code can't do anything with that and logs "AdaptorService.install failed". That's LIGHTNING-WF: 643 occurrences since February 2025, still going.
• @latest. The entry's version is the literal word latest, which isn't a version number, so the comparison raises instead. That's LIGHTNING-1JE: 13 occurrences since March, still going.

Same cause, and which one you get depends only on whether the job pinned its adaptor.

The good news is it's contained to that one adaptor. The lookup checks the name before it checks the version, so a half-installed entry doesn't disturb anything else.

To see it

Two overlapping installs of an adaptor that isn't present yet:

spec = "@openfn/language-http@7.3.2"
Task.async(fn -> Lightning.AdaptorService.install(:adaptor_service, spec) end)
Process.sleep(400)
Lightning.AdaptorService.install(:adaptor_service, spec)

The second call returns {:ok, %Adaptor{status: :installing, path: nil}}, which is what produces the warning. Swap the pinned version for @latest and it raises Version.InvalidVersionError with version "latest" instead.

Worth knowing why this comes up so often: a server starts with only a handful of adaptors present, so installs aren't unusual, they're the normal path, and it starts over each time it restarts.

Why it started when it did

Every job defaults to @openfn/language-common@latest, so a great many jobs carry the word latest rather than a pinned version.

The old editor never sent that word to the adaptor service. It resolved latest into a real version number first, so the code that can't cope with it was never asked to. Restoring metadata in the new editor (#4388, 19 March) dropped that step: the channel now reads the adaptor straight off the job and passes it along as-is. The first LIGHTNING-1JE error is six days later.

The pinned-version half doesn't depend on that change, which fits it going back to February 2025.

A separate, smaller thing

LIGHTNING-22Q fired once, on 11 August, where the version asked for was the word local. That one needs a job with local actually saved against it, and we found five across two workflows on production, both since deleted. They came from internal work rather than a customer. We do check the adaptor field on write, both for shape and that the package name is one the registry knows, but the version part is deliberately permissive: the pattern explicitly allows latest and local, and in practice any word gets through. How local came to be saved on those particular jobs is still unaccounted for. It isn't from provisioning, a template or a sandbox copy, we checked all three.

Related history: when we added local adaptor support we started labelling those adaptors with the version local and never taught the metadata side about it, so metadata for local adaptors has never actually worked. Joe hit that in February in #3647.

Impact

Runs aren't affected at all. There's no retry loop. The person loses autocomplete on that job and we get a Sentry alert.

Done when

• A half-installed adaptor is never visible to a lookup, which removes both faults at once.
• A version we can't parse never raises anywhere in this path. It comes back as a normal error the channel already handles, so the person sees a clean "couldn't load metadata" and we don't get an alert. This is the safety net rather than the fix.
• latest is resolved to a real version before it reaches the adaptor service, or the adaptor service handles it properly itself.
• Tests cover this, which today they don't at all. There is a test file for this module but it only covers shell injection, the allowlist and package-name parsing. Wanted: two overlapping installs, and versions latest, local, something arbitrary like next, and empty.
• LIGHTNING-WF, LIGHTNING-1JE and LIGHTNING-22Q all stop appearing.

Start at Lightning.AdaptorService.install and follow the metadata lookup path, then inspect the existing test file for this module. Reproduce the overlapping-install example and exercise pinned, latest, local, arbitrary, and empty versions. Done means half-installed entries are hidden, invalid versions become handled errors, latest is resolved or supported, and regression tests cover the cases.