OpenFn / OpenFn/lightning

Run export history as a background process/task

Open
#1,765 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Elixir
Stars
296
Forks
86
Avg merge
1d 13h
Merged PRs (30d)
50

Description

The export worker offloads export operations to the background.

  • Initiate history export functionality in the background when a user requests an export.

  • Triggers sending email to project admin and user who made the request when export is completed or if an error occurred

  • Allow history export based on filters to the export functionality based on workflow, status, received date, last activity, and search string.

  • Completed exports are stored in a bucket defined in the storage service.

  • The data should be exported in a .json structure considering different work orders have varying number of steps.

  • Naming convention: Lightning_history_export_{{uuid_of_export}}.json??

  • Specified download path should be a signed URL i.e. download(path) => signed url

  • Relevant status of exports are :completed, :in_progress, :failed, :expired.

  • Set expiry_date of the file to a 24 hour period after the history was exported.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by locating the export worker and storage service, then trace how history filters and export requests are handled. Done requires background execution, completion or failure emails, filtered JSON files in the configured bucket, signed download URLs, the listed statuses, and 24-hour expiry.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
backend, cloud, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.