elsa-workflows / elsa-workflows/elsa-core

[Tracking] Large bookmark backlog slows scheduling startup (partially mitigated; residual #8155/#8156, #7524/#7730)

Open
#7,470 3 comments 0 reactions 0 assignees View on GitHub
triaged
Dominant language
C#
Stars
7.9k
Forks
1.5k
Avg merge
15h 22m
Merged PRs (30d)
114

Description

## Description
When a large amount of bookmarks (25k+ in our case) are pending in the database and the application is restarted, the startup takes a long time before it's ready to receive requests. This is caused by the Tenant Activator triggering the Bookmark scheduler in a HostedService's StartAsync() function.

## Steps to Reproduce
1. Queue a large amount of bookmarks (in my case via a Delay activity)
2. Restart the application

Callstack:
ActivateTenants.cs -> StartAsync()
DefaultTenantService.cs
-> ActivateTenantsAsync()
-> RefreshAsync()
-> RegisterTenantAsync()
TenantEventsManager.cs
-> TenantActivatedAsync()
-> ExecuteEventHandlersAsync()
UpdateTenantSchedules.cs
-> TenantActivatedAsync()
DefaultBookmarkScheduler.cs
-> ScheduleAsync()

ScheduleAsync() has a foreach that goes over each element sequentially.

## Expected Behavior
I expect the re-scheduling of the bookmarks to not block the application startup and happen in the background.

## Actual Behavior
The application startup is paused until all the bookmarks are rescheduled.

## Environment
- **Elsa Package Version**: 3.6.2
- **Operating System**: Windows 11

## Troubleshooting Attempts
Describe any steps you have taken to troubleshoot or resolve the issue yourself.

## Additional Context
Add any other context about the problem here. This could include the frequency of the issue (e.g., intermittent, every time), conditions under which the issue appears, etc.

## Related Issues
Link to any related issues here.

Contributor guide

Open the contributing guide

Research direction

Start at ActivateTenants.StartAsync() and follow the listed activation path through DefaultTenantService, TenantEventsManager, UpdateTenantSchedules, and DefaultBookmarkScheduler.ScheduleAsync(). Reproduce with a large pending bookmark backlog and inspect the sequential scheduling behavior. Done means application startup can receive requests without waiting for all bookmarks to be rescheduled, while the residual cases in #8155, #8156, #7524, and #7730 remain covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.