Azure / Azure/durabletask

Tasks are running even when orchestration is complete

Open
#1,137 11 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
1.7k
Forks
335
Avg merge
2d 23h
Merged PRs (30d)
6

Description

Hi,
I am using DTF with Azure Storage Provider.

I have an orchestration which contains several tasks. The orchestration completes successfully. However I see that there are 2 tasks which are hidden in the workItem queue. Even after the orchestration is complete, these tasks keep on executing forever.

I tried to debug from logs to understand what is going on here, but could not go further.
Below is an example of a good task (task#3) and a bad task (task#5) which executes forever.

LifeStory of Task#3

Sending [TaskScheduled#3] message to newprovisioninghub-workitems for instance 'c2102a9a4d5942e4a1bf91e0767b50c8'

Fetched [TaskScheduled#3] message from newprovisioninghub-workitems (delay = 45ms)

Processing [TaskScheduled#3] (total delay = 45ms)

.......... Runs the Task - <>

Sending [TaskCompleted#3] message to newprovisioninghub-control-00 for instance 'c2102a9a4d5942e4a1bf91e0767b50c8'

Deleting [TaskScheduled#3] message from newprovisioninghub-workitems

Fetched [TaskCompleted#3] message from newprovisioninghub-control-00 (delay = 90ms)

Processing [TaskCompleted#3] (total delay = 166ms)

Deleting [TaskCompleted#3] message from newprovisioninghub-control-00

---------------------

LifeStory of Task#5

Sending [TaskScheduled#5] message to newprovisioninghub-workitems for instance 'c2102a9a4d5942e4a1bf91e0767b50c8'

Fetched [TaskScheduled#5] message from newprovisioninghub-workitems (delay = 0ms)

Processing [TaskScheduled#5] (total delay = 0ms)

newprovisioninghub-control-00: No new messages were found - backing off

Renewing [TaskScheduled#5] message on newprovisioninghub-workitems for an additional 1200 seconds

Sending [TaskCompleted#5] message to newprovisioninghub-control-00 for instance 'c2102a9a4d5942e4a1bf91e0767b50c8'

Deleting [TaskScheduled#5] message from newprovisioninghub-workitems

Fetched [TaskCompleted#5] message from newprovisioninghub-control-00 (delay = 251ms)

Failed to update or delete message [TaskScheduled#5] with ID = c4eb20ac-48be-47d2-bf25-dd202a67c8e4 because it no longer exists

Processing [TaskCompleted#5] (total delay = 335ms)

Deleting [TaskCompleted#5] message from newprovisioninghub-control-00

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.