open-telemetry / open-telemetry/opentelemetry-python

Clients requests with partial set of B3 propagation headers causes crash

Open
#2,629 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug propagators
Dominant language
Python
Stars
2.6k
Forks
1k
Avg merge
4d 15h
Merged PRs (30d)
19

Description

Describe your environment
Python 3.8
Flask 2.0.3
opentelemetry-propagator-b3 = 1.10.0
opentelemetry-instrumentation-flask = =0.29b0

Steps to reproduce
Following https://opentelemetry.io/docs/instrumentation/python/getting-started/#configure-your-http-propagator-b3-baggage, set_global_textmap using the B3MultiFormat() rather than the deprecated B3Format.

set_global_textmap(B3MultiFormat())

Make an API request from Postman (or other client tool of your choice), and set the following headers:

  • x-b3-traceid : 8b65d983c41ee5a6
  • x-b3-spanid : 8b65d983c41ee5a6
  • b3 : 8b65d983c41ee5a6-8b65d983c41ee5a6

See that it works

Make the same request, but this time do not set the b3 header (still set the x-b3-traceid and x-b3-spanid headers)

See that it crashes

Make the same request without any of the headers,

See that it works

What is the expected behavior?
Clients providing incomplete set of propagation headers should not result in an unhandled exception being raised to the web app.

What is the actual behavior?

2022-04-20T19:23:47.632614+00:00 [ERROR] [None] [5368] waitress - Exception while serving <path omitted>
Traceback (most recent call last):
  File "<path-omitted>\.venv\lib\site-packages\waitress\channel.py", line 426, in service
    task.service()
  File "<path-omitted>\.venv\lib\site-packages\waitress\task.py", line 168, in service
    self.execute()
  File "<path-omitted>\.venv\lib\site-packages\waitress\task.py", line 434, in execute
    app_iter = self.channel.server.application(environ, start_response)
  File "<path-omitted>\.venv\lib\site-packages\paste\translogger.py", line 69, in __call__
    return self.application(environ, replacement_start_response)
  File "<path-omitted>\.venv\lib\site-packages\flask\app.py", line 2091, in __call__
    return self.wsgi_app(environ, start_response)
  File "<path-omitted>\.venv\lib\site-packages\werkzeug\middleware\proxy_fix.py", line 187, in __call__
    return self.app(environ, start_response)
  File "<path-omitted>\.venv\lib\site-packages\opentelemetry\instrumentation\flask\__init__.py", line 170, in _wrapped_app
    return wsgi_app(wrapped_app_environ, _start_response)
  File "<path-omitted>\.venv\lib\site-packages\flask\app.py", line 2080, in wsgi_app
    return response(environ, start_response)
  File "<path-omitted>\.venv\lib\site-packages\werkzeug\wrappers\response.py", line 630, in __call__
    start_response(status, headers)
  File "<path-omitted>\.venv\lib\site-packages\opentelemetry\instrumentation\flask\__init__.py", line 156, in _start_response
    if span.kind == trace.SpanKind.SERVER:
AttributeError: 'NonRecordingSpan' object has no attribute 'kind'

Process finished with exit code 0

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with opentelemetry/instrumentation/flask/init.py, especially _wrapped_app and _start_response from the traceback. Reproduce the request using x-b3-traceid and x-b3-spanid without the b3 header, then compare it with the complete-header and no-header cases. Done means incomplete propagation headers no longer raise an unhandled exception in the Flask application.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, observability-sre
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.