Spans not recorded by default if context is propagated

Describe your environment

OS: Ubuntu 2204
Python version: 3.10.12
Package version: 1.3.0/0.51b0

What happened?

I originally raised this issue as https://github.com/open-telemetry/opentelemetry-python-contrib/issues/3267 but now realise it seems to be a core bug rather than in a particular instrumentor.

The root of this issue appears to be that the propagator in https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-api/src/opentelemetry/trace/propagation/tracecontext.py#L85-L87 returns a NonRecordingSpan if traceparent is set, however if it is not set a default _Span is created. https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py#L1160 then converts this into an _Span iff sampler is specified and returns that we should sample this trace.

The net result is some very confusing (and I think undocumented) behaviour: By default the root span is recording no matter whether a sampler is used or not. But if a valid traceparent header is seen it defaults to not recording unless there is a sampler set and it returns True (eg OTEL_TRACES_SAMPLER=always_on)

Steps to Reproduce

from fastapi import FastAPI
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor

resource = Resource.create()
trace.set_tracer_provider(TracerProvider(resource=resource))
span_processor = BatchSpanProcessor(ConsoleSpanExporter())
trace.get_tracer_provider().add_span_processor(span_processor)

app = FastAPI()
FastAPIInstrumentor.instrument_app(app)


@app.get("/")
def root():
    return {"message": "Hello World"}

Run as fastapi dev v.py

Expected Result

Test with:

curl http://127.0.0.1:8000
curl http://127.0.0.1:8000 -H "traceparent: 00-94df63b03874c83da3cc8e207789df94-17fb4659f79f7ca2-00"

I would expect this to log to 2 traces to the console.

Actual Result

Only the first trace is logged, the one with traceparent is dropped.

Run again with OTEL_TRACES_SAMPLER=always_on fastapi dev v.py and repeat the curls and both traces appear on the console.

Additional context

https://github.com/open-telemetry/opentelemetry-python-contrib/issues/3267

Would you like to implement a fix?

None

Start with the propagation logic in opentelemetry-api/src/opentelemetry/trace/propagation/tracecontext.py around lines 85-87 and the span creation logic in opentelemetry-sdk/src/opentelemetry/sdk/trace/init.py around line 1160. Run the FastAPI reproduction with and without the traceparent header, then trace how sampling affects recording. Done means both curl requests produce recorded console spans under the default configuration.