OpenFn / OpenFn/lightning

Better Adaptor List Refreshing

Open
#2,209 3 comments 0 reactions 1 assignee View on GitHub

@taylordowns2000 is already working on this.

Since Feb 27, 2026.

For Consideration
Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

Current Issue:
The adaptor list is only loaded once at server startup, requiring restarts to see new adaptors or versions. This is increasingly problematic as we add pre-release versions and more adaptors.

Proposed Solution:
Implement a dual strategy approach:

  1. Use Cachex for high-concurrency access to the adaptor list
  2. Implement a Strategy pattern with:
    • A clear behavior contract for retrieving adaptors
    • Separate implementations for NPM Registry and Local Repository sources
  3. Add a periodic refresh mechanism that runs in the background
  4. Keep the data source configuration options we already have

Benefits:

  • No server restarts needed to see new adaptor versions
  • Better performance for concurrent adaptor list access
  • Clear separation of concerns between data retrieval and caching
  • Easily extensible if we need to add other adaptor sources
  • Compatible with both NPM registry and local development setups

Implementation Approach:
We'll refactor the existing AdaptorRegistry into a more modular system while maintaining backward compatibility. The registry will determine the appropriate strategy when starting, then periodically refresh using that same strategy in the background.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.