Fork-Safety Issue Causing EBADF and Telemetry Data Loss

Describe your environment

OS: RHEL 8
Python version: 3.12
SDK version: 1.25.0
API version: 1.25.0

What happened?

In fork-based, daemonized workloads, OpenTelemetry initializes a persistent HTTP session in the parent process. When the process later forks and aggressively closes inherited file descriptors (FD hygiene), the child process retains a weakref.finalize callback created in the parent. During interpreter shutdown, this finalizer attempts to close an already-closed socket, resulting in OSError: [Errno 9] Bad file descriptor. Additionally, telemetry export may silently stop working in the child due to stale sessions.

Steps to Reproduce

> cat /var/tmp/l.py
#!/usr/bin/env python3
import errno
import os
import resource
import socket
import sys
import threading
import time
from http.server import BaseHTTPRequestHandler, HTTPServer

from opentelemetry import metrics
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader


MAXFD = 2048


def get_maximum_file_descriptors():
    soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
    if hard == resource.RLIM_INFINITY:
        return MAXFD
    return hard


def close_file_descriptor_if_open(fd: int) -> None:
    try:
        os.close(fd)
    except OSError as exc:
        if exc.errno != errno.EBADF:
            raise


def close_all_open_files(exclude=None) -> None:
    exclude = set(exclude or {0, 1, 2})
    maxfd = get_maximum_file_descriptors()
    for fd in range(maxfd):
        if fd in exclude:
            continue
        try:
            close_file_descriptor_if_open(fd)
        except Exception as exc:
            print(f"[child] error closing fd {fd}: {exc}", file=sys.stderr)


def detach_process_context() -> None:
    pid = os.fork()
    if pid > 0:
        os._exit(102)
    os.setsid()


class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def do_POST(self):
        length = int(self.headers.get("Content-Length", "0"))
        if length:
            _ = self.rfile.read(length)
        self.send_response(200)
        self.send_header("Content-Length", "0")
        self.end_headers()

    def log_message(self, fmt, *args):
        pass


def start_local_http_server():
    server = HTTPServer(("127.0.0.1", 0), Handler)
    host, port = server.server_address
    thread = threading.Thread(target=server.serve_forever, daemon=True)
    thread.start()
    return server, port


def main() -> None:
    server, port = start_local_http_server()
    endpoint = f"http://127.0.0.1:{port}/v1/metrics"
    print(f"[parent] local OTLP-like HTTP server on {endpoint}")

    exporter = OTLPMetricExporter(endpoint=endpoint)
    reader = PeriodicExportingMetricReader(
        exporter,
        export_interval_millis=1000,
        export_timeout_millis=1000,
    )
    provider = MeterProvider(metric_readers=[reader])
    metrics.set_meter_provider(provider)

    meter = metrics.get_meter("fork-fd-repro")
    counter = meter.create_counter("fork_fd_repro_counter")

    print("[parent] recording metrics")
    for i in range(3):
        counter.add(1, {"iteration": i})
        time.sleep(1.2)

    # By now, exporter session/pool should definitely be exercised.
    print("[parent] forking once")
    detach_process_context()

    print("[child] closing all file descriptors")
    close_all_open_files()

    print("[child] exiting normally")
    return


if __name__ == "__main__":
    main()

> python /reproducer.py
[parent] local OTLP-like HTTP server on http://127.0.0.1:45515/v1/metrics
[parent] recording metrics
[parent] forking once
[child] closing all file descriptors
[child] exiting normally
Traceback (most recent call last):
  File "/opt/python/python-3.11/lib64/python3.11/weakref.py", line 666, in _exitfunc
    f()
  File "/opt/python/python-3.11/lib64/python3.11/weakref.py", line 590, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python/python3/std/lib64/python3.11/site-packages/urllib3/connectionpool.py", line 1180, in _close_pool_connections
    conn.close()
  File "/usr/local/python/python3/std/lib64/python3.11/site-packages/urllib3/connection.py", line 272, in close
    super().close()
  File "/opt/python/python-3.11/lib64/python3.11/http/client.py", line 981, in close
    sock.close()   # close it manually... there may be other refs
    ^^^^^^^^^^^^
  File "/opt/python/python-3.11/lib64/python3.11/socket.py", line 503, in close
    self._real_close()
  File "/opt/python/python-3.11/lib64/python3.11/socket.py", line 497, in _real_close
    _ss.close(self)
OSError: [Errno 9] Bad file descriptor

Expected Result

OpenTelemetry should be safe to use in fork-based workloads.
After fork():
Parent-owned resources (sessions, sockets, background threads, finalizers) should not be reused in the child.
Telemetry in the child process should either:
Be cleanly reinitialized, or
Be explicitly disabled unless reinitialized by the application.
No errors should be emitted during interpreter shutdown.

Actual Result

No error

Additional context

Root Cause Analysis
OpenTelemetry creates a requests.Session in the parent process.
requests.Session registers a weakref.finalize callback to close sockets at interpreter shutdown.
After fork:
The child inherits the finalizer callback.
The batch runner closes all inherited FDs at the OS level.
At shutdown, the inherited finalizer attempts to close an FD that has already been closed, resulting in EBADF.
This is a classic fork-safety issue where cleanup logic for parent-owned resources runs in the child process.

Would you like to implement a fix?

Yes

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

Start by running the supplied /reproducer.py example and tracing the exporter session, inherited finalizer, and fork lifecycle described in the traceback. No repository file or test is named, so locate the relevant OpenTelemetry metrics/exporter entry points before deciding on the design. Done means forked children do not reuse parent resources, child telemetry is cleanly reinitialized or disabled, and shutdown emits no EBADF error.