microsoft / microsoft/durabletask-java

`TaskOrchestrationContext.waitForExternalEvent` Timeout "Removed" After Attempting to Schedule Duplicate Orchestration

Open
#203 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

P2
Dominant language
Java
Stars
29
Forks
18
Avg merge
1d 10h
Merged PRs (30d)
2

Description

Summary

An orchestration which specifies a timeout for waiting on an external event will seemingly have that timeout "removed"/ignored after another attempt is made to schedule an orchestration instance with the same ID.

Steps to Reproduce

  1. Copy the example below.
  2. Execute the orchestrator-trigger function via HTTP.
@FunctionName("orchestrator-trigger")
public HttpResponseMessage triggerOrchestrator(
    @HttpTrigger(
        name = "request",
        methods = { HttpMethod.GET },
        authLevel = AuthorizationLevel.FUNCTION,
        route = "orchestrator/trigger"
    )
    HttpRequestMessage<String> request,
    @DurableClientInput(name = "durableContext")
    DurableClientContext durableContext,
    ExecutionContext context
) throws InterruptedException {
    DurableTaskClient client = durableContext.getClient();
    String instanceId = "the-only-instance";

    client.scheduleNewOrchestrationInstance("orchestrator", null, instanceId);
    client.raiseEvent(instanceId, "first", 1);

    // Attempting to schedule another orchestration instance with the same instance ID results
    // in the event timeout within the existing orchestrator instance from triggering.
    try {
        client.scheduleNewOrchestrationInstance("orchestrator", null, instanceId);
    } catch (RuntimeException ignored) { }

    Thread.sleep(10_000);
    client.raiseEvent(instanceId, "second", 2);

    return request.createResponseBuilder(HttpStatus.OK).build();
}

@FunctionName("orchestrator")
public void orchestrator(
    @DurableOrchestrationTrigger(name = "orchestration")
    TaskOrchestrationContext orchestration,
    ExecutionContext context
) {
    Task<Integer> firstTask =
        orchestration.waitForExternalEvent("first", Duration.ofSeconds(1), Integer.class);

    Task<Integer> secondTask =
        orchestration.waitForExternalEvent("second", Duration.ofSeconds(1), Integer.class);

    List<Integer> results = orchestration.allOf(firstTask, secondTask).await();
    int first = results.get(0);
    int second = results.get(1);

    System.out.printf("Triggered! First: %s, Second: %s\n", first, second);
}

Expected Result

The orchestration throws an exception due to the second event not arriving within one second.

Actual Result

After ten seconds, the orchestration prints Triggered! First: 1, Second: 2.

Additional Context

Deleting the entire try/catch block that contains the second scheduleNewOrchestrationInstance call results in the expected outcome - the orchestration throws.

This test case is contrived - my actual use case is:

  • A process operates on the combination of a ZIP file and a CSV file.
  • These two files are provided to the application separately, in any order, but at roughly the same time.
  • Whichever file arrives first needs to start an orchestration instance that will wait for both files.
  • We need to avoid a race condition that would result in two separate orchestration instances being created, each waiting for the opposite file. So we'll create a deterministic instance ID based on other data, have the CSV/ZIP receivers always attempt to schedule an orchestration instance with that ID, and ignore the "already exists" error when it occurs.
  • The CSV/ZIP receivers then simply send their own single event to the orchestration instance.
  • The orchestration instance needs to have a timeout in the situation where the second file never arrives.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Java reproduction in the issue, focusing on TaskOrchestrationContext.waitForExternalEvent and the duplicate scheduleNewOrchestrationInstance call. Run the orchestrator-trigger scenario and verify that the second external event times out after one second even when scheduling the same instance ID is attempted again.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.