Cyclomatic complexity survey: codebase is healthy overall, with a few concentrated hot spots

Summary

I ran a cyclomatic complexity analysis over the repository using lizard (covering the Rust, Python, Scala, and Java sources) and wanted to share the results, since they may be useful for prioritizing refactoring and for readers navigating the code.

Headline numbers:

• 13,417 functions analyzed; average cyclomatic complexity (CCN) is 3.3 - well under the conventional threshold of 10.
• 822 functions exceed CCN 10, but 371 of those are generated Thrift serialization code (read_from_in_protocol / write_to_out_protocol in thunder/schema/*.rs and similar). Excluding generated code, only 451 of 12,100 hand-written functions (~3.7%) are over the threshold.
• The feed-serving path most readers come to this repo for is the cleanest part of the codebase: home-mixer averages CCN 2.0 and visibility-filtering averages 1.6, each with only a handful of functions over 10.

Per-component breakdown (hand-written code only, sorted by average CCN)

All remaining components (vm-ranker, scarecrow, agatha, clip, media-model-proxy, candidate-pipeline, visibility-filtering-client, user-cred-v2, adult-content, pnsfwmedia) average under 3.2 with few or no functions over the threshold.

Most complex hand-written functions

Observations

1. Complexity concentrates in ML infrastructure, not ranking logic. The largest hot spots are Phoenix's serving engine, its hand-tuned flash-attention kernels in phoenix/xrex/cutedsl/ranker_fa4/ (where heavy branching is largely inherent to GPU kernel programming), and training checkpoint handling. The scoring, filtering, and visibility logic that determines what users see is simple and readable.

2. Two functions stand out as genuine refactoring candidates:

   • prefetcher_loop in bdsm/runtime/batch_prefetcher.py - a single 566-line loop at CCN 136.
   • compute_batch_inputs_rust_quick in phoenix/crates/serving/xai-recsys-engine/src/python.rs - 682 lines at CCN 115, plus a second overload at CCN 61 in the same file.

   Both mix several responsibilities in one body and would benefit from being split into smaller units; for a repo whose stated goal is transparency and readability, these are the files where a reader is most likely to get lost.

3. Generated code dominates the raw warning count. If the project ever adds a complexity linter to CI, excluding */schema/* and other Thrift-generated sources would keep the signal useful.

Reproducing

pip install lizard
lizard -l rust -l python -l scala -l java --CCN 10 \
  -x "*/target/*" -x "*/node_modules/*" -x "*/.git/*"

Start by rerunning the documented lizard command, then inspect bdsm/runtime/batch_prefetcher.py and phoenix/crates/serving/xai-recsys-engine/src/python.rs, especially the named high-complexity functions. The report identifies refactoring candidates but does not define a specific scope or completion criteria, so a follow-up would need to select one target and establish its tests before changes.