Azure / Azure/azure-functions-host

Flex Consumption graceful-shutdown e2e (E18) depends on host shutdown log strings; find a more reliable signal

Open
#11,864 1 comment 0 reactions 0 assignees View on GitHub
Needs: Attention :wave:
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 10h
Merged PRs (30d)
36

Description

## Summary

The Flex Consumption graceful-shutdown e2e test (E18) asserts that the .NET Functions Host ran its graceful-shutdown / telemetry-flush chain by grepping for specific host shutdown **log strings**. This coupling is fragile: if the host ever changes those log messages, the match will no longer succeed and the test will start failing (or need manual updating) even though the behavior it validates is still correct.

This came out of review of `Forward SIGTERM to Functions Host process group on shutdown`, which fixes intermittent tail-telemetry loss on instance recycle by forwarding SIGTERM to the host so its `ApplicationStopping -> StopAsync/Dispose -> App Insights/OTel flush` chain runs.

## Where the coupling lives

The test lives in the `AAPT-Antares-Functions-Docker` repo, but the strings it depends on are **owned by this host repo**:

- `images/flexconsumption/e2e-test/e2e-harness.sh` (`test_E18_sigterm_forwarded_to_host`), regex:
```
Stopping host\.\.\.|Host shutdown completed\.|Host state changed from Stopping to Stopped
```
- The same list is mirrored in prose at `images/flexconsumption/components/appserver/cv2/graceful-shutdown-design.md`.

If the host renames/reformats any of these shutdown log messages, the e2e assertion silently stops matching.

## Severity

Low / non-blocking. The failure mode is **safe**: a changed log string makes the marker count fail to increase, so E18 **fails loudly** rather than producing a false pass. So this is about brittleness and maintenance, not a correctness gap.

## Asks

1. **Evaluate more reliable signals** for "the host shut down gracefully and flushed telemetry" that don't depend on exact log text, e.g.:
- a stable framework-level marker (`Microsoft.Hosting.Lifetime` "Application is shutting down..."), or
- the host process **exit code** (graceful SIGTERM shutdown exits 0 vs SIGKILL 137), or
- a documented/contractual shutdown log line the host commits to keeping stable.
2. **In the meantime, keep a regression test in place** so the graceful-shutdown/flush-on-SIGTERM behavior can't regress, and treat any of the strings above as a known dependency (update the e2e regex + design doc together if host shutdown logging changes).

## References

- PR tracked in internal issue link

Contributor guide

Open the contributing guide

Research direction

Start with images/flexconsumption/e2e-test/e2e-harness.sh, especially test_E18_sigterm_forwarded_to_host, and compare its shutdown-log regex with the mirrored discussion in images/flexconsumption/components/appserver/cv2/graceful-shutdown-design.md. Evaluate the listed framework marker, process exit code, and contractual log-line options, then preserve a regression check for graceful SIGTERM shutdown and document any remaining dependency.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, shell
Domain
backend, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.