Protocol: microbe co-occurrence and mutual exclusivity across signatures

Tier: C (BugSigDB paper) · Type: atomic · Category: Statistical Analysis

What

Tally how often pairs of microbes appear together within signatures in the same direction versus in opposite directions, normalize the resulting matrix, and contrast it against microbe prevalence and pairwise correlation in healthy samples.

Why it matters

It is the one analysis in the paper that finds structure between microbes rather than between studies, and the contrast against healthy-sample correlation is what separates a real ecological signal from "both are common". The vignette also carries a worked warning about scaling — 100% self-co-occurrence swamping the color scale — which is a normalization decision that belongs in a protocol, not a comment.

Source material

• waldronlab/BugSigDBPaper — vignettes/Figure4.Rmd (increased/decreased tallies, same-direction and opposite-direction matrices, the colMaxs scaling and its dampening comment, "Contrasting with microbe prevalence in healthy samples", "Contrasting with correlation in healthy samples", "Identify conditions associated with certain genera")
• Paper: 10.1038/s41587-023-01872-y

Scope

In: the pair-counting rule; same- versus opposite-direction matrices; the normalization and the self-co-occurrence problem; the healthy-sample reference set and the Spearman correlation contrast; phylum-level annotation; what a co-occurrence claim is and is not evidence for.

Out: visualization specifics beyond what the normalization requires.

Frontmatter starting point

type: "atomic"
category: "Statistical Analysis"
citation: "10.1038/s41587-023-01872-y"   # Possibly correct for the signature-level tally, but
                                         # co-occurrence analysis has a long ecology literature
                                         # behind it. Check before accepting.
tags: [bugsigdb, co-occurrence, mutual-exclusivity, correlation, ecology]

Acceptance criteria

• The normalization is specified, including the self-pair handling
• The healthy reference set is a named, pinned input
• Carries an explicit caution against causal or interaction interpretation

Cite the method's origin, not its users

PROTOCOL_STANDARD.md is explicit: an atomic protocol carries "strictly 1 citation... corresponding
to the primary literature where the method was originally published." Find the paper that proposed
the method. Do not cite a paper that merely applied it — including the BugSigDB and curatedMetagenomicData
papers, which are the source of the analysis these protocols were extracted from but almost never the
source of the method.

Tracing a method back to its first publication is real work, and it is part of the task, not a
formality. Three things to expect:

• Candidate DOIs in this issue are leads, not answers. Anything marked VERIFY has not been checked.
• Some methods predate modern citation practice or have no single identifiable origin. If that is
  genuinely the case, say so in the pull request rather than reaching for a convenient recent paper.
  Raise it as an issue in waldronlab/agent-protocol-standard — the standard may need a way to express
  "classical method, no primary source".
• If you cannot name one paper that proposed everything the protocol does, it is more than one
  protocol. That test has now split four protocols out of this batch: enrichment into three methods,
  filtering from transformation, LODO from random forest, and PERMANOVA from ANOSIM.

Where the lab's own paper genuinely did propose the method — the oral-to-gut score, and LODO
cross-validation in Pasolli et al. 2016 — citing it is correct. That is the exception, not the pattern.

Before you start

Read CONTRIBUTING.md and
PROTOCOL_STANDARD.md.
The format is defined in the standard repo, not this one. Protocols are prose, not code: they say what to do
and why, precisely enough that two people — or two agents, in two languages — get the same answer. The existing
independent-filtering-variance
protocol is the model to imitate for tone and level of detail.

Validate locally before opening the PR:

git clone https://github.com/waldronlab/agent-protocol-standard.git
Rscript agent-protocol-standard/scripts/validate-protocol.R protocols

Read CONTRIBUTING.md and PROTOCOL_STANDARD.md, then use protocols/independent-filtering-variance/protocol.md as the prose model. Inspect Figure4.Rmd for the tallying, normalization, healthy-sample reference, and correlation analyses, and identify the primary paper that proposed the method. Validate the completed protocol with Rscript agent-protocol-standard/scripts/validate-protocol.R protocols; done means all three acceptance criteria are explicit.