Bug: Granian `--no-subprocess` mode crashes on macOS when using Litestar logger

🐛 Granian --no-subprocess mode crashes on macOS when using Litestar logger

Environment

• OS: macOS (darwin, aarch64)
• Python: 3.13.12
• Granian: via litestar-granian 0.14.2
• Litestar: 2.19.0
• multiprocessing start method: spawn (macOS default since Python 3.8)

Problem

Running the Litestar app with LITESTAR_GRANIAN_IN_SUBPROCESS=false causes the Granian worker to crash immediately on startup.
The worker spawns but exits immediately with no error message. This happens on macOS and only when both conditions are met:

1. LITESTAR_GRANIAN_IN_SUBPROCESS=false
2. LITESTAR_GRANIAN_USE_LITESTAR_LOGGER=true

Attempted workaround: forcing fork — also fails

The project already has a workaround in src/py/app/__init__.py that forces the fork multiprocessing method on macOS:

# src/py/app/__init__.py
import multiprocessing
import platform

if platform.system() == "Darwin":
    multiprocessing.set_start_method("fork", force=True)

However, this does not fix the issue — it merely changes the failure mode from a silent pickle error to a Segmentation fault:

Fatal Python error: Segmentation fault

Current thread 0x00000001fbea58c0 (most recent call first):
  File ".../granian/server/mp.py", line 64 in wrapped
  File ".../multiprocessing/process.py", line 108 in run
  File ".../multiprocessing/process.py", line 313 in _bootstrap
  File ".../multiprocessing/popen_fork.py", line 74 in _launch
  ...

Extension modules: greenlet._greenlet, msgspec._core, sqlalchemy.cyextension.*,
  PIL._imaging, uvloop.loop, psycopg_binary._psycopg, asyncpg.protocol.protocol,
  hiredis.hiredis, setproctitle._setproctitle (total: 22)

The segfault occurs because macOS's Objective-C runtime and numerous C extension modules (greenlet, uvloop, asyncpg, psycopg, PIL, hiredis, etc.) are not fork-safe when threads are already running. By the time Granian spawns workers, the main process has already started the Vite dev server subprocess, logging queue listeners, and potentially other background threads — making fork unsafe.

Summary of both failure modes:

Start method	Failure	Cause
spawn (macOS default)	Silent crash — "Unexpected exit from worker-1"	log_dictconfig contains unpicklable closures/instances
fork (forced via set_start_method)	Segmentation fault	C extensions + Obj-C runtime not fork-safe with active threads

Root Cause Analysis

TL;DR

On macOS, in_subprocess=False is a dead end regardless of the multiprocessing start method:

• spawn (default): The Litestar logger configuration dict contains unpicklable Python objects (closures, class instances), causing a silent pickle failure when passed through multiprocessing.Process.
• fork (forced): The main process has already loaded 22+ C extension modules and started background threads (Vite, logging queues), making fork trigger a Segmentation fault in the child process.

Detailed Walkthrough

1. Two modes of Granian startup

The LITESTAR_GRANIAN_IN_SUBPROCESS environment variable controls how Granian is launched:

Mode	Code path	How workers get config
true (default)	_run_granian_in_subprocess() → subprocess.Popen(["python", "-m", "granian", ...])	Command-line strings — no pickle needed
false	_run_granian() → Granian(log_dictconfig=...).serve() → multiprocessing.Process(target, args)	pickle serialization — must be serializable

2. macOS forces spawn multiprocessing

# Confirmed on the affected system:
>>> import multiprocessing
>>> multiprocessing.get_start_method()
'spawn'

The spawn method creates a fresh Python interpreter for each worker and uses pickle to transfer the target function and all arguments. This is different from Linux's fork, which copies the parent's memory space directly.

Granian's worker process code (granian/server/mp.py):

class WorkerProcess(AbstractWorker):
    def __init__(self, parent, idx, target, args):
        self._spawn_method = multiprocessing.get_start_method()
        if self._spawn_method not in {'fork', 'spawn'}:
            self._spawn_method = 'spawn'
        # ...

    def _spawn(self, target, args):
        self.inner = multiprocessing.get_context(self._spawn_method).Process(
            name='granian-worker', target=target, args=args
        )

3. The smoking gun: unpicklable log_dictconfig

When LITESTAR_GRANIAN_USE_LITESTAR_LOGGER=true, the CLI calls _get_logging_config(env, use_litestar_logger=True) which returns a dict containing:

{
    'formatters': {
        'standard': {
            '()': <class 'structlog.stdlib.ProcessorFormatter'>,  # class object
            'processors': [
                <structlog.processors.TimeStamper object>,         # instance
                <function add_log_level>,                          # function
                <structlog.stdlib.ExtraAdder object>,              # instance
                <app.lib.log.EventFilter object>,                  # custom instance
                <structlog.processors.EventRenamer object>,        # instance
                <structlog.processors.JSONRenderer object>,        # instance
                # ...
            ]
        }
    },
    # ...
    'exception_logging_handler': <function _default_exception_logging_handler_factory.<locals>._default_exception_logging_handler>
    # ^^^ THIS IS A CLOSURE — cannot be pickled!
}

Verification:

>>> import pickle
>>> pickle.dumps(log_dictconfig)
# AttributeError: Can't get local object
#   '_default_exception_logging_handler_factory.<locals>._default_exception_logging_handler'

This dict is passed to Granian(log_dictconfig=...), which stores it and later passes it as an argument to multiprocessing.Process. On macOS with spawn, this triggers the pickle failure.

4. Why it fails silently

When multiprocessing.Process with spawn fails to unpickle arguments in the child process, the child exits with code 1. The parent (Granian main process) only sees "Unexpected exit from worker-1" — the actual AttributeError is lost because it occurs during the child's bootstrap phase before any logging is configured.

5. Why in_subprocess=True works

In subprocess mode, Granian is launched as a completely separate process via:

subprocess.Popen([sys.executable, "-m", "granian", env.app_path, *cli_args])

All configuration is passed as command-line string arguments. The logging config is handled by Granian's own CLI parsing, which uses a simple default dict — no pickle involved.

6. Why Uvicorn never hits this issue

Litestar's default Uvicorn runner has two paths:

• Single worker: uvicorn.run(app=app_path) — runs in the current process, no multiprocessing
• Multi-worker/reload: subprocess.run(["python", "-m", "uvicorn", ...]) — subprocess with CLI args

Neither path ever passes log_dictconfig through multiprocessing.Process, so pickle serialization is never required.

Verification Matrix

Test scenario	Start method	Result	Reason
Granian direct (no Litestar CLI, no logger config)	spawn	✅ Works	Default log config is plain strings, picklable
CLI + in_subprocess=true + use_litestar_logger=true	N/A (subprocess)	✅ Works	Config passed as CLI strings, no pickle
CLI + in_subprocess=false + use_litestar_logger=false	spawn	✅ Works	Default log config is picklable
CLI + VitePlugin lifespan + Granian direct	spawn	✅ Works	No Litestar logger config involved
CLI + in_subprocess=false + use_litestar_logger=true	spawn	❌ Silent crash	log_dictconfig contains unpicklable objects
CLI + in_subprocess=false + use_litestar_logger=true	fork (forced)	❌ Segfault	22 C extensions + threads make fork unsafe

Solutions

Workaround 1: Disable Litestar logger when using --no-subprocess (recommended for debugging)

LITESTAR_GRANIAN_IN_SUBPROCESS=false \
LITESTAR_GRANIAN_USE_LITESTAR_LOGGER=false \
uv run app run

Or in __main__.py:

os.environ.setdefault("LITESTAR_GRANIAN_IN_SUBPROCESS", "false")
os.environ.setdefault("LITESTAR_GRANIAN_USE_LITESTAR_LOGGER", "false")  # Must be false!

Workaround 2: Use remote debugging with subprocess mode

Keep in_subprocess=true (default) and attach a debugger to the worker process:

# In create_app() or a startup hook:
import debugpy
debugpy.listen(("0.0.0.0", 5678))
# debugpy.wait_for_client()  # Uncomment to pause until debugger attaches

Then attach VS Code or PyCharm to port 5678.

Proper Fix (library level)

The _get_logging_config() function in litestar-granian should sanitize the logging config dict to ensure pickle compatibility when in_subprocess=False. Specifically:

1. Replace class/instance references with importable string paths (e.g., "structlog.stdlib.ProcessorFormatter" instead of <class ...>)
2. Replace closures with module-level function references
3. Or: skip passing log_dictconfig to Granian entirely when in non-subprocess mode, and configure logging directly in the current process instead

Alternatively, Granian could detect unpicklable log_dictconfig and fall back to configuring logging in the main process before spawning workers (which the spawn method would then inherit via re-import).

Related

• macOS switched multiprocessing default from fork to spawn in Python 3.8 (python/cpython#84559)
• Python 3.14 will default to forkserver on Linux, making this class of bugs more widespread
• Granian already handles the Python 3.14 change: if self._spawn_method not in {'fork', 'spawn'}: self._spawn_method = 'spawn'

---

Reference: The --in-subprocess/--no-subprocess option was introduced in commit 002ff92 by @cofin, authored on Jan 8.
Also see #245 Bug: project not starting on Mac Sequioa 15.7.3

Start with the Granian startup paths described in the issue, especially _get_logging_config and the src/py/app/init.py multiprocessing workaround. Reproduce with LITESTAR_GRANIAN_IN_SUBPROCESS=false and LITESTAR_GRANIAN_USE_LITESTAR_LOGGER=true on macOS, then verify the worker starts without the silent spawn failure or fork-related crash.