OpenFn / OpenFn/lightning

Better coupling of portability spec with kit

Open
#4,734 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs technical design
Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

The Portability Spec is the standard for how a project is represented in a serializable format. Basically it's the project spec, or project.yaml file.

In the past we haven't been very good at keeping this aligned across the CLI and Lightning. But we've recently done work to fix this.

Something on my mind now is that the definition of the project spec is a bit fragmented:

  • kit defines a project spec file in the lexicon package
  • Lightning has a JSON schema to define the spec
  • And I think Lightning might have its own typings for the generator code?

We really should consolidate this. Consider:

  • Generating a JSON schema from a single type definition (the json schema is great for external interopability)
  • Re-using packages from kit and lightning
  • Maybe the yaml generator code in lightning should actually be using @openfn/project directly?
  • Moving any portability spec code into its own standalone repo, so it's easier to share and reason about.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Compare the project spec definition in kit's lexicon package with Lightning's JSON schema and generator typings, including the proposed use of @openfn/project. Start by mapping where each definition is consumed; done should mean the portability spec has one clear source of truth and its schema and generator consumers remain aligned.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.