sillsdev / sillsdev/TheCombine

Investigate a job system for our long-running async tasks (project export & duplicate finding)

Open
#3,744 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backend goal: MergeDup import/export performance
Dominant language
TypeScript
Stars
22
Forks
10
Avg merge
2d 2h
Merged PRs (30d)
9

Description

https://github.com/sillsdev/TheCombine/blob/v2.3.1/Backend/Controllers/LiftController.cs#L342 and https://github.com/sillsdev/TheCombine/pull/3725/files#diff-67204fb92eb948fe05297ee9457a502af06f6c0535590666113ee19fa7379032 both use _ = Task.Run(() => ... to kick off an async task that might take several minutes to complete, then will use SignalR to signal the frontend upon completion.

@hahn-kev observes that sometimes "the pod/container is moved to another k8s node (basically the server restarts)" and "I'd look into a job system which means the job is saved in the DB to ensure it runs even if there's a restart." and "We use Quartz.net".

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Backend/Controllers/LiftController.cs at line 342 and the changes in pull request 3725 to identify the long-running tasks that use Task.Run and SignalR. Review the existing Quartz.net usage and determine the job-system design needed to persist work in the database and survive a Kubernetes restart; the issue is complete when that approach is defined for project export and duplicate finding.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, kubernetes
Domain
backend, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.