OpenFn / OpenFn/lightning

Send the worker more details in the run payload

Open
#3,750 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

DevX
Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

A Run sent to the Worker (the thing in the returned by the fetch:plan event) currently looks like this:

{
  "dataclip_id": "f0bd6d3f-a4d1-4678-b1ec-6788fc3c65fb",
  "edges": [
    {
      "condition": "always",
      "enabled": true,
      "id": "33dce70f-047f-4508-82fd-950eb508519b",
      "source_job_id": null,
      "source_trigger_id": "3b4a47c0-7242-4f0c-8886-838e34762654",
      "target_job_id": "b8b780f3-98dd-4244-880b-e534d8f24547"
    }
  ],
  "id": "68d0bb3f-86d7-48c7-bee7-305f9d4c3381",
  "jobs": [
    {
      "adaptor": "@openfn/language-common@3.1.0",
      "body": "log(state.configuration.name.toUpperCase())",
      "credential_id": "9b5e7cc3-47ef-4e4b-ac81-993bc9d9ef10",
      "id": "b8b780f3-98dd-4244-880b-e534d8f24547",
      "name": "Transform data"
    }
  ],
  "options": {
    "run_timeout_ms": 300000,
    "output_dataclips": true
  },
  "starting_node_id": "b8b780f3-98dd-4244-880b-e534d8f24547",
  "triggers": [
    {
      "id": "3b4a47c0-7242-4f0c-8886-838e34762654"
    }
  ]
}

It would be great to add a few more fields for the worker:

  • workflowId: the UUID of the workflow
  • workflowName: the human readable name of the workflow
  • workflowSlug: The worker can generate this from the workflow name, but it might be useful for lightning to generate this itself

It might make sense to add snapshot metadata too? The version number maybe?

This isn't mission critical, but would help us add extra metadata to runs

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace the fetch:plan event and the Run payload sent to the Worker, then identify where workflow metadata is available. Confirm the expected fields and whether snapshot metadata is in scope; done means the requested workflow identifiers are included in the payload with coverage for the new shape.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.