OpenFn / OpenFn/lightning

Update adaptors on the fly

Open
#4,771 0 comments 0 reactions 1 assignee View on GitHub

@stuartc is already working on this.

Since May 19, 2026.

Adaptor Registry Epic
Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

Hypothesis

If new adaptors, versions, icons, and credential schemas appear in Lightning
without an operator restart, then services and sales teams can use them as
soon as they're released to NPM — and clustered deployments stop showing
stale or inconsistent versions to users.

Problem observations

  • A new adaptor (or new version of an existing one) requires a Lightning
    restart to appear in the UI. Sales and services are blocked on releases.
  • Even after restart, adaptor icons sometimes don't appear on the workflow
    diagram — they rely on a separate build step.
  • Credential schemas are baked in at build time. A new adaptor schema isn't
    available until the next deploy.
  • In clustered deployments, nodes can disagree about which adaptor versions
    exist, causing the version dropdown to silently downgrade a job's adaptor.
  • The adaptor list, icon set, and credential schemas are each loaded by a
    different mechanism, so any fix has to address three subsystems at once.

In scope

  • Adaptor registry refreshes automatically on a schedule, without restart.
  • New adaptors and new versions appear across all nodes in a cluster
    consistently.
  • Credential schemas are fetched from NPM at runtime and kept current.
  • Adaptor icons are fetched and served at runtime; pre-baked icons still
    work in airgapped/offline deployments.
  • Superusers can trigger a manual refresh from a Settings → Maintenance page
    when they don't want to wait for the schedule.
  • The LOCAL_ADAPTORS mode (local filesystem source) continues to work for
    development and airgapped deployments, with no upstream NPM calls.

Out of scope

  • Defining credential schemas that aren't linked to a specific adaptor
    (separate concern — see notes on #325 and #220 below).
  • Per-project adaptor version pinning.
  • Content-addressed icon URLs / long-lived browser caching of icons.

Sub-issues

Notes on what each contributes and what (if anything) is left for later:

  • #2209 — periodic background refresh of the adaptor list, no restart
    needed. Fully addressed.
  • #3114 — new adaptors' icons and schemas appear without a release.
    Both the icons-without-restart and schemas-without-restart parts are
    addressed.
  • #1996 — cluster nodes disagree about adaptor versions. Addressed via
    a shared source-of-truth across nodes.
  • #325 (2022) — pull credential schemas from NPM. The runtime-fetch
    part is addressed; the question of where to define non-adaptor-linked
    schemas is not.
  • #220 (2022) — credential types list pre-populated and the caching
    strategy question. The caching/refresh part is addressed; non-adaptor
    credential types are not.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.