API Background tasks created with asyncio.create_task could be garbage collected

Describe the bug

Background tasks created with asyncio.create_task are not managed or retained

Description

Background tasks started during the FastAPI application lifespan are created using asyncio.create_task() but are not stored or managed. This results in tasks being unreferenced, which can lead to premature garbage collection, lost exceptions, and uncontrolled shutdown behaviour.

Problem

The code previously used:

asyncio.create_task(deploymentStatusUpdater.receive_messages())
asyncio.create_task(airlockStatusUpdater.receive_messages())

Issues with this approach:

• No strong references to tasks
  • Tasks may be garbage collected while still running
• No lifecycle management
  • Tasks are not cancelled or awaited during shutdown
• Unobserved exceptions
  • Failures inside tasks are silently ignored
• Potential shutdown warnings
  • e.g. Task was destroyed but it is pending!

Impact

• Background workers may terminate unexpectedly
• Errors in long-running tasks can go unnoticed
• Application shutdown may be unreliable or produce warnings
• Difficult to debug production issues due to lack of visibility

Expected behaviour

• Background tasks should:
  • Be strongly referenced
  • Be tied to the FastAPI application lifecycle
  • Be cancelled and awaited on shutdown
  • Have exceptions logged with full context

Suggested fix

Track and manage tasks via app.state, for example:

app.state.background_tasks = set()

task = asyncio.create_task(worker())
app.state.background_tasks.add(task)
task.add_done_callback(app.state.background_tasks.discard)

And on shutdown:

tasks = list(app.state.background_tasks)
for task in tasks:
    task.cancel()

await asyncio.gather(*tasks, return_exceptions=True)

Additional notes

• Tasks should also handle asyncio.CancelledError internally for clean shutdown.
• Naming tasks can significantly improve observability and debugging.

---

Steps to reproduce

I have not noticed any times this has happened in practice

Azure TRE release version (e.g. v0.14.0 or main):
main
Deployed Azure TRE components - click the (i) in the UI:
latest

Locate the FastAPI application lifespan code containing the two asyncio.create_task calls and inspect how startup and shutdown are handled. Implement the expected task retention, cancellation, awaiting, and exception visibility described in the issue, including clean handling of asyncio.CancelledError. Done means the workers remain referenced during runtime and shut down without pending-task warnings.