Azure / Azure/azure-functions-host

Process for host instance notification shouldn't rely on file watchers

Open
#10,861 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

design needs-investigation
Dominant language
C#
Stars
2k
Forks
483
Avg merge
2d 10h
Merged PRs (30d)
36

Description

This item is to track investigation into alternative methods for coordination across host instances, such as a queue based model, etc.

Today, file-based sentinels are used, but this has a few issues:

  1. This can be missed in situations where the assumption of a shared file system isn't upheld. That can manifest as a silent failure for the scenario, though the impact would be lessened by other coordinated recycle events such as configuration changes to the resource.
  2. Using file-based also creates a requirement on a mutable filesystem being present, which limits hosting environment options.
  3. File watchers have a material perf impact. We should strive to move away from them.

An example scenario can be seen with the default secrets repository: https://github.com/Azure/azure-functions-host/blob/ed038bed405924d4736b5448025c8a518a0ad48f/src/WebJobs.Script.WebHost/Security/KeyManagement/BlobStorageSecretsRepository.cs#L140-L141

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the default secrets repository example in src/WebJobs.Script.WebHost/Security/KeyManagement/BlobStorageSecretsRepository.cs, especially the referenced lines, and trace how file-based sentinels and watchers coordinate host instances. Investigate queue-based or other alternatives against the shared-filesystem, mutable-filesystem, and performance concerns. Done means an agreed replacement approach is documented or implemented with its coordination tradeoffs established.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.