Tasks are running even when orchestration is complete
- Dominant language
- C#
- Stars
- 1.7k
- Forks
- 335
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 6
Description
Hi,
I am using DTF with Azure Storage Provider.
I have an orchestration which contains several tasks. The orchestration completes successfully. However I see that there are 2 tasks which are hidden in the workItem queue. Even after the orchestration is complete, these tasks keep on executing forever.
I tried to debug from logs to understand what is going on here, but could not go further.
Below is an example of a good task (task#3) and a bad task (task#5) which executes forever.
LifeStory of Task#3
Sending [TaskScheduled#3] message to newprovisioninghub-workitems for instance 'c2102a9a4d5942e4a1bf91e0767b50c8'
Fetched [TaskScheduled#3] message from newprovisioninghub-workitems (delay = 45ms)
Processing [TaskScheduled#3] (total delay = 45ms)
.......... Runs the Task - <>
Sending [TaskCompleted#3] message to newprovisioninghub-control-00 for instance 'c2102a9a4d5942e4a1bf91e0767b50c8'
Deleting [TaskScheduled#3] message from newprovisioninghub-workitems
Fetched [TaskCompleted#3] message from newprovisioninghub-control-00 (delay = 90ms)
Processing [TaskCompleted#3] (total delay = 166ms)
Deleting [TaskCompleted#3] message from newprovisioninghub-control-00
---------------------
LifeStory of Task#5
Sending [TaskScheduled#5] message to newprovisioninghub-workitems for instance 'c2102a9a4d5942e4a1bf91e0767b50c8'
Fetched [TaskScheduled#5] message from newprovisioninghub-workitems (delay = 0ms)
Processing [TaskScheduled#5] (total delay = 0ms)
newprovisioninghub-control-00: No new messages were found - backing off
Renewing [TaskScheduled#5] message on newprovisioninghub-workitems for an additional 1200 seconds
Sending [TaskCompleted#5] message to newprovisioninghub-control-00 for instance 'c2102a9a4d5942e4a1bf91e0767b50c8'
Deleting [TaskScheduled#5] message from newprovisioninghub-workitems
Fetched [TaskCompleted#5] message from newprovisioninghub-control-00 (delay = 251ms)
Failed to update or delete message [TaskScheduled#5] with ID = c4eb20ac-48be-47d2-bf25-dd202a67c8e4 because it no longer exists
Processing [TaskCompleted#5] (total delay = 335ms)
Deleting [TaskCompleted#5] message from newprovisioninghub-control-00
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.