Composite protocol: systematic review of BugSigDB signatures for one condition

Tier: A (capstone-scale) · Type: composite · Category: Statistical Analysis

What

The end-to-end fieldwork analysis, as a composite protocol: subset BugSigDB for a condition → study characteristics table → taxon frequency and direction test → Monte Carlo critical frequency → signature similarity and clustering → interpretation.

Why it matters

This is the deliverable that makes the whole exercise worth it. A student picks a new condition, runs one composite protocol, and produces a complete, comparable mini-review — instead of copying someone else's .Rmd and editing the filter. It is also the direct test of whether the Tier A atomic protocols actually compose.

Source material

Six existing worked examples of this same workflow, each a candidate validation case:

Scope

In: ordering and hand-off between the component protocols; the condition-selection and inclusion criteria; what the final report must contain; how to state limitations (publication bias, curation coverage, non-independence of signatures from the same study).

Out: everything the component protocols already specify — do not restate their steps.

Frontmatter starting point

type: "composite"
category: "Statistical Analysis"
citation: "10.1038/s41587-023-01872-y"
protocols_used:
  - name: "bugsigdb-signature-subset"
  - name: "signature-study-table"
  - name: "taxon-frequency-direction-binomial"
  - name: "montecarlo-critical-frequency"
  - name: "jaccard-signature-similarity"
  - name: "signature-similarity-clustering"

The similarity measure is the composite's choice point: swap jaccard-signature-similarity for
taxonomic-semantic-similarity and the rest of the composition is unchanged. State which one this
composite specifies and why, rather than leaving it open.

Acceptance criteria

• Depends on the six Tier A atomic protocols; contains no duplicated steps
• Reproduces the headline findings of at least one existing vignette above when run on the same condition
• Blocked on the six Tier A protocols merging first

Cite the method's origin, not its users

PROTOCOL_STANDARD.md is explicit: an atomic protocol carries "strictly 1 citation... corresponding
to the primary literature where the method was originally published." Find the paper that proposed
the method. Do not cite a paper that merely applied it — including the BugSigDB and curatedMetagenomicData
papers, which are the source of the analysis these protocols were extracted from but almost never the
source of the method.

Tracing a method back to its first publication is real work, and it is part of the task, not a
formality. Three things to expect:

• Candidate DOIs in this issue are leads, not answers. Anything marked VERIFY has not been checked.
• Some methods predate modern citation practice or have no single identifiable origin. If that is
  genuinely the case, say so in the pull request rather than reaching for a convenient recent paper.
  Raise it as an issue in waldronlab/agent-protocol-standard — the standard may need a way to express
  "classical method, no primary source".
• If you cannot name one paper that proposed everything the protocol does, it is more than one
  protocol. That test has now split four protocols out of this batch: enrichment into three methods,
  filtering from transformation, LODO from random forest, and PERMANOVA from ANOSIM.

Where the lab's own paper genuinely did propose the method — the oral-to-gut score, and LODO
cross-validation in Pasolli et al. 2016 — citing it is correct. That is the exception, not the pattern.

Before you start

Read CONTRIBUTING.md and
PROTOCOL_STANDARD.md.
The format is defined in the standard repo, not this one. Protocols are prose, not code: they say what to do
and why, precisely enough that two people — or two agents, in two languages — get the same answer. The existing
independent-filtering-variance
protocol is the model to imitate for tone and level of detail.

Validate locally before opening the PR:

git clone https://github.com/waldronlab/agent-protocol-standard.git
Rscript agent-protocol-standard/scripts/validate-protocol.R protocols

Read CONTRIBUTING.md and PROTOCOL_STANDARD.md first, then inspect the six named Tier A protocols, the six example Rmd files, and independent-filtering-variance/protocol.md. Define the composite's ordering, hand-offs, inclusion criteria, similarity choice, citations, and limitations without duplicating atomic steps. Validate with the provided Rscript command and reproduce the headline findings of at least one example after the six dependencies merge.