open-telemetry / open-telemetry/opentelemetry-python-contrib

FastApi Opentelemetry throwing concurrent.futures._base.CancelledError

Open
#467 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triaged
Dominant language
Python
Stars
1.1k
Forks
1.1k
Avg merge
4d 15h
Merged PRs (30d)
16

Description

For Kubernetes I have implemented healthchecks in FastAPI like this

@router.get("/health/liveness", tags=["health_checks"])
async def liveness():
    if check_liveness():
        result = {"status": "UP"}
    else:
        result = {"result": "DOWN"}
    return result

However, opentelemetry always gives errors in these spans. Not sure why. This is the error I get in Datadog Tracing

CancelledError
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/opentelemetry/trace/__init__.py", line 522, in use_span
yield span
File "/usr/local/lib/python3.7/site-packages/opentelemetry/sdk/trace/__init__.py", line 879, in start_as_current_span
yield span_context
File "/usr/local/lib/python3.7/site-packages/opentelemetry/instrumentation/asgi/__init__.py", line 213, in wrapped_receive
message = await receive()
File "/usr/local/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 526, in receive
await self.message_event.wait()
File "/usr/local/lib/python3.7/asyncio/locks.py", line 293, in wait
await fut
concurrent.futures._base.CancelledError

And this happens for all the endpoints. Not sure whats up. Any help is appreciated.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ASGI instrumentation in opentelemetry/instrumentation/asgi/init.py, especially wrapped_receive, and reproduce the cancellation using the FastAPI health-check endpoint shown. Review existing ASGI instrumentation tests for cancellation behavior; done means the reported CancelledError is understood and handled or documented consistently for instrumented endpoints.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, python
Domain
api, backend, observability-sre
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.