OpenFn / OpenFn/kit

Project: rethink from-fs tests

Open
#1,171 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

DevX
Dominant language
TypeScript
Stars
21
Forks
23
Avg merge
1d 22h
Merged PRs (30d)
17

Description

A really important function, from-fs, is responsible for loading a Project definition from whatever is checked out on the file system.

The complexity here is that there are two sources of truth: the project/state file (which should represent the last fetch from the app), and the local file system.

When I do deploy, the from-fs function needs to cleverly load the active project from the workflows file, reconcile it against the server to apply uuids and options and stuff, and then push it. This is a lot like a merge really - what happens if the user changed ids in workflow.yaml? We need to track as best as possible.

Should we literally treat this as a force merge? Take the local system, merge it in-memory onto the local project file, then push the merged file up to the app? That probably makes sense because I can feel I'm about to duplicate a lot of that logic.

Anyway, the problems are that this function doesn't actually work super great, and the tests are awful.

Improvements needed to the tests:

  • Include v2 and v2 state files
  • Really thorough testing that all any information in the state that is not contained in the workflow (ie, the openfn object) gets correctly transferred

This is the issue behind #1168 . I've worked around it but I'd like to rethink this when I have more time

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the from-fs entry point and inspect the existing tests alongside the v2 and v2 state-file cases. Add thorough coverage for transferring state information not present in workflow.yaml, including the openfn object; completion should address the behavior described in relation to #1168.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
cli, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.