[Product validation] Distribution gate: distinguish no demand from no reach

Decision this issue exists to protect

ContextWeaver currently has weak evidence of independent retained use. That is important, but absence of adoption is not by itself evidence that the product problem is weak if distribution has never been competently tested.

This issue adds a distribution-quality gate to #758/#840 so the survival review can distinguish:

1. no reach — qualified users did not meaningfully encounter/evaluate the product;
2. positioning/onboarding failure — they encountered it but did not understand or activate;
3. product/value failure — they activated but did not retain it;
4. narrow pull — they retain one small capability and ignore the broader architecture;
5. real pull — independent users retain the workflow and would notice its removal.

The goal is not to manufacture adoption. The goal is to make non-adoption interpretable.

Revised survival principle

Do not use current user count, stars, forks, package downloads or maintainer-created integrations as a kill signal. First expose a small number of sharp product hypotheses to qualified users through competent distribution, then measure activation and retention.

This does not protect the current architecture. Capabilities already duplicated adequately by provider/framework-native mechanisms can still be removed on competitive/maintenance grounds. Distribution is a confounder only for hypotheses whose incremental value remains genuinely uncertain.

Hypotheses to expose

Test no more than three sharply scoped propositions. Do not build three products.

D1 — capability snapshot + semantic drift detection

Promise:

Capture the effective capability surface of an agent, commit it, and see semantically meaningful changes before deployment.

Candidate workflow:

contextweaver snapshot ...
contextweaver inspect snapshot.json
contextweaver diff baseline.json candidate.json
contextweaver verify candidate.json

This is the lowest-integration-cost hypothesis and should be the primary distribution test.

D2 — bounded / phase-aware context compilation

Promise:

Give the model only the capability/context needed for the current phase, deterministically and reproducibly.

This must be tested against modern provider-native context/tool mechanisms. Interest alone is insufficient: require a measurable reason to keep the extra layer (quality, reproducibility, controllability, cost, debugging, or another concrete outcome).

D3 — deterministic tool selection beyond native tool search

Treat this primarily as a falsification experiment, not a presumed product pillar.

Compare against current provider-native tool search/deferred loading and a simple retrieval baseline. Keep it only if qualified users or comparative evidence reveal a material advantage that matters in practice.

What counts as competent distribution

A hypothesis is not allowed to fail the demand gate until all of the following are true for the tested proposition:

• one painfully concrete problem statement, not an architecture-led description;
• a copy/paste quickstart aimed at first useful result in roughly 10 minutes or less;
• at least one real-source example using a mainstream capability source (MCP and/or OpenAPI initially);
• the user can reach the value without understanding Weaver Stack or ContextWeaver internals;
• setup does not require a model/provider account when the tested value is offline;
• explicit comparison with the obvious alternative (ordinary Git/config/tests or provider-native mechanisms);
• targeted distribution reaches qualified practitioners rather than a generic AI audience;
• at least three distinct acquisition paths are attempted, including direct outreach to plausible users;
• responses are tracked as exposure → visit/read → attempted activation → first success → retained/removed;
• negative evidence and reasons for non-use are preserved.

Suggested acquisition paths can include direct maintainer/platform-engineer outreach, relevant technical communities, GitHub discussions/issues where participation is appropriate, and the maintainer's existing professional/content channels. Do not spam or manufacture engagement.

Minimum experiment cohort

This issue complements, rather than replaces, the design-partner cohort in #840.

For at least the primary D1 proposition, aim for:

• 10–20 qualified direct exposures where we can reasonably believe the person saw the proposition;
• ~5 genuine evaluation attempts if the exposure pool produces them;
• unassisted attempts in addition to high-touch design partners;
• follow-up sufficient to distinguish one-off curiosity from retained workflow use.

These are experiment targets, not vanity KPI thresholds. A small number of strong retained users is more informative than a large number of clicks.

Funnel evidence

Record counts/reasons at these boundaries:

qualified exposure
  → understood the problem
  → chose to evaluate
  → attempted setup
  → reached first useful output
  → used on a real project
  → retained independently

For every meaningful drop-off, capture the reason when available:

• problem not relevant/severe;
• existing alternative already sufficient;
• proposition unclear;
• setup/integration too expensive;
• output not useful;
• useful only as one-off diagnostic;
• useful narrow slice, broader product unnecessary;
• missing blocker/feature;
• trust/security/process concern;
• other.

Interpretation rules

Distribution failure / inconclusive

Do not infer weak demand when qualified exposure was too small or the proposition/onboarding was clearly poor. Fix distribution/positioning once before interpreting demand.

Positioning/onboarding failure

Qualified users engage with the problem but cannot understand or reach the promised value. Simplify the proposition/workflow; do not add architecture.

Product/value failure

Qualified users repeatedly reach first success but remove/abandon the workflow because the alternative is cheaper or the value is not consequential. This is strong shrink/delete evidence.

Narrow product signal

Users repeatedly retain one capability while bypassing the rest. Treat the retained slice as the candidate product and delete pressure increases on the rest.

Strong survival signal

Independent users retain the workflow on genuine projects, can use it without maintainer operation, and can state what painful/manual/risky process it replaced or materially improved.

Kill / shrink conditions after competent distribution

For a tested hypothesis, strong negative evidence exists when:

• qualified practitioners understand the proposition but consistently decline because the problem is low severity or already solved cheaply;
• evaluators reach first value but do not keep it;
• retention depends on maintainer operation or bespoke integration;
• the only valued outcome is a commodity capability already available natively;
• users retain only a narrower slice, making the broader architecture unnecessary.

If D1 fails after competent distribution and D2/D3 also lack strong comparative pull, #758 should move toward archive/major shrink rather than inventing a fourth thesis.

Architecture freeze during the experiment

Until this gate has interpretable evidence:

• no new framework adapters without a concrete evaluator blocker;
• no new routing sophistication without comparative failure evidence;
• no new memory/session/gateway surface justified by speculative adoption;
• no new store/backend breadth without a real integration requirement;
• prefer deleting or parking clearly commoditized/duplicated surfaces;
• keep implementation work focused on making the tested proposition small, understandable, reliable and easy to evaluate.

Acceptance criteria

• D1 has a concrete distribution-ready proposition and unassisted quickstart;
• qualified exposure is measured rather than inferred from public availability;
• at least three acquisition paths are exercised, including direct targeted outreach;
• activation and first-success evidence are separated from traffic/stars/downloads;
• retention/removal evidence is collected where elapsed time permits;
• non-adoption reasons are recorded and classified;
• #840 synthesis distinguishes demand failure from distribution/positioning/onboarding failure;
• #758 survival decision uses retained value after competent exposure, not raw absence of users;
• if the evidence is negative, the response is shrink/archive rather than another speculative product pivot.

Related

• #758 — controlling product thesis / survival decision
• #840 — neutral design-partner discovery
• #658 — unassisted onboarding evidence
• #551 — genuine adoption / retention evidence
• #841 — claims/evidence integrity; distribution material must not reuse suspect benchmark claims
• #445 — comparative evidence against contemporary baselines

Start with the D1 capability snapshot proposition and compare its acceptance criteria with #840, #758, and the unassisted onboarding evidence in #658. Define the experiment around qualified exposure, activation, first useful output, retention, and recorded non-adoption reasons; done means the evidence can distinguish distribution failure from product or value failure without relying on stars or downloads.