Protocol: random-effects meta-analysis with the Paule-Mandel heterogeneity estimator

Tier: D (cMD paper) · Type: atomic · Category: Statistical Analysis

What

Pool per-dataset effect sizes and standard errors into a random-effects summary using the Paule-Mandel estimator of between-study variance, reporting the combined effect, its standard error and confidence interval, a Z statistic and p-value, tau², Cochran's Q and I², with dependency-robust FDR correction across features.

Why it matters

It is the statistical core of the cMD paper and is currently implemented twice — once in R via meta::metagen and once as a hand-rolled Python class hierarchy — which is exactly the situation a language-agnostic protocol is for. Reconciling the two implementations against one written specification is itself a worthwhile student project and may well surface a discrepancy.

Source material

• waldronlab/curatedMetagenomicDataAnalyses — R/runMetaanalysis_categorical.R and R/runMetaanalysis_quantitative.R (meta::metagen, method.tau = "PM", sm = "SMD"), R/filter_sort.R (the na_filter minimum-cohort rule), python_modules/meta_analyses.py (paule_mandel_tau, RE_meta, RE_meta_binary, tau_squared_DL)
• Paper: 10.1038/s41467-025-66888-1 — Paule-Mandel; Benjamini-Yekutieli FDR at 0.01/0.1; Q and I² heterogeneity

Scope

In: the Paule-Mandel iteration and its convergence criteria (the Python implementation uses atol=1e-5, maxiter=50); weighting; the minimum number of contributing cohorts before a feature is reportable; Benjamini-Yekutieli correction and why it rather than Benjamini-Hochberg; Q and I² definitions; what to report when tau² is zero.

Out: computing the input effect sizes (component protocols).

Frontmatter starting point

type: "atomic"
category: "Statistical Analysis"
citation: "10.6028/jres.087.022"   # Paule & Mandel 1982 — verify before merging
tags: [meta-analysis, random-effects, paule-mandel, heterogeneity, fdr]

Acceptance criteria

• The PM iteration is specified including convergence tolerance and iteration cap
• The minimum-cohort filter is a stated parameter (it currently lives in filter_sort.R)
• The choice of Benjamini-Yekutieli over Benjamini-Hochberg is justified in ## Notes
• The R and Python implementations agree on a shared test case, or the discrepancy is documented

---

Cite the method's origin, not its users

PROTOCOL_STANDARD.md is explicit: an atomic protocol carries "strictly 1 citation... corresponding
to the primary literature where the method was originally published." Find the paper that proposed
the method. Do not cite a paper that merely applied it — including the BugSigDB and curatedMetagenomicData
papers, which are the source of the analysis these protocols were extracted from but almost never the
source of the method.

Tracing a method back to its first publication is real work, and it is part of the task, not a
formality. Three things to expect:

• Candidate DOIs in this issue are leads, not answers. Anything marked VERIFY has not been checked.
• Some methods predate modern citation practice or have no single identifiable origin. If that is
  genuinely the case, say so in the pull request rather than reaching for a convenient recent paper.
  Raise it as an issue in waldronlab/agent-protocol-standard — the standard may need a way to express
  "classical method, no primary source".
• If you cannot name one paper that proposed everything the protocol does, it is more than one
  protocol. That test has now split four protocols out of this batch: enrichment into three methods,
  filtering from transformation, LODO from random forest, and PERMANOVA from ANOSIM.

Where the lab's own paper genuinely did propose the method — the oral-to-gut score, and LODO
cross-validation in Pasolli et al. 2016 — citing it is correct. That is the exception, not the pattern.

Before you start

Read CONTRIBUTING.md and
PROTOCOL_STANDARD.md.
The format is defined in the standard repo, not this one. Protocols are prose, not code: they say what to do
and why, precisely enough that two people — or two agents, in two languages — get the same answer. The existing
independent-filtering-variance
protocol is the model to imitate for tone and level of detail.

Validate locally before opening the PR:

git clone https://github.com/waldronlab/agent-protocol-standard.git
Rscript agent-protocol-standard/scripts/validate-protocol.R protocols

Read CONTRIBUTING.md, PROTOCOL_STANDARD.md, and protocols/independent-filtering-variance/protocol.md first. Compare R/runMetaanalysis_categorical.R, R/runMetaanalysis_quantitative.R, R/filter_sort.R, and python_modules/meta_analyses.py, then verify the method's primary citation. Done means a protocol covering PM convergence, cohort filtering, BY correction, heterogeneity reporting, and an agreed or documented shared test case; run the supplied validate-protocol.R command.