Multi-phase activities

A lot of real life activities have multiple phases that require different timeouts. For example a human task needs to be inserted into an external system with a pretty short timeout and then be picked up by a human within another timeout. The third timeout specifies how long the activity can be worked up after it was claimed.

Current solution to the above use case is to use an activity to insert the task into the external system. And then signal workflow about each task state change. While this workaround works it significantly complicates workflow code especially having that timeouts should be enforced in the business logic through timers.

The proposal is to model an activity execution as a list of phases with each phase having its own timeout.

There are multiple ways to achieve this. Here are two options:

1. Specify list of (phase, timeout, retryPolicy) triples when scheduling an activity. Allow retrieving the current value of the triple when retrying an activity. Add an additional API to complete a phase or augment CompleteActivityTask with additional fields to use it for phase completion.

2. Treat phases as an internal activity detail. In this case an activity can override heartbeat timeout as part of a heartbeat. This way it can store whatever information needed in the progress field and change heartbeat timeout to conform with the current phase requirements. For example in the case of human task the initial heartbeat timeout is going to be as small as needed to insert the task into the external system, then it is going to be extended to the maximum queue time, and so on.

No files or tests are named. Start by tracing the activity scheduling, retry, heartbeat, and completion APIs described in the proposal, then compare the two phase-model options against existing workflow behavior. Done requires an agreed design and an implementation that supports distinct timeouts across activity phases.