[Bulk index] Replace symbol-by-symbol LSP scans with compiler-owned snapshots

Outcome

Replace declaration-by-declaration LSP crawling with one versioned graph-snapshot
contract implemented at the semantic database that already owns each language.
The result must satisfy both halves of the product requirement:

1. export every fact that the selected compiler/analyzer universe can determine,
   with uncertainty and unsupported families represented explicitly; and
2. pay at most one native semantic-analysis cost for a cold repository, then
   make unchanged and ordinary edit refreshes shard-based and near-instant.

This is not a request to tune lspConcurrency, add another cache around
textDocument/references, or treat SCIP as a complete graph format.

Current master: what has already landed

PR #147, merged as
95e20c9,
completed the provider registry, truthful authority/fact declarations,
generation fencing, strict real-tool experiments, and exact cold-index
publication. The former foundation issues #64-#71 and #85 are closed. Do not
rebuild those foundations.

The reusable implementation is already present in:

• IGraphProvider,
  including authority, facts, build inputs, configuration, preparation, refusal,
  resolution, and session opening;
• IBulkGraphSession,
  including atomic snapshots, source/checker digests, provenance, and refresh
  modes;
• BatchGraphSession
  and the resident project transaction, which already reject stale, malformed,
  cancelled, or crashed generations without replacing the prior graph;
• ISamchonGraphDump.IProvenance,
  which already records provider, languages, authority, registered fact
  families, producer versions, universe, manifest digest, and content digest;
  and
• strict lifecycle and real-producer checks in
  tests/experiment.

The remaining gap is not provider discovery. It is the producer channel,
per-generation completeness contract, incremental shard reuse, and preservation
of that truth through the MCP response.

Fact-checked performance baseline

The committed publication
tests/benchmark/results/graph.json
came from the exact-head index-time run
30448033020.
These are cold end-to-end cells on pinned repositories, four visible cores and
16 GiB RAM; they are not resident no-op or edit measurements.

Language / corpus	Strict-provider cell	Strict-off fallback cell
TypeScript / Excalidraw	5.340 s	2.978 s static
Go / Gin	38.097 s	0.687 s static
Python / Flask	10.629 s	0.748 s static
Java / Gson	88.653 s	231.398 s
C / Redis	22.795 s	262.906 s
C++ / LevelDB	8.353 s	26.452 s
PHP / Slim	3.772 s	9.611 s
C# / Serilog	20.498 s	25.085 s
Kotlin / Koin	211.264 s	967.712 s
Lua / Lualine	18.889 s	27.848 s
Rust / Tokio	55.238 s	229.861 s
Ruby / Sinatra	stopped at 1,800 s	stopped at 1,800 s
Dart / dart-http	stopped at 1,800 s	stopped at 1,800 s

The Ruby and Dart result is deliberately narrow: both cells were terminated by
the 1,800,000 ms guard, so the publication establishes that neither cell
finished within 30 minutes. It does not report their eventual completion
time, and it does not isolate the strict producer's time from later fallback
work. In both strict logs, the run first attempted scip-ruby/scip-dart and
was eventually aborted while an LSP textDocument/references request was
active. The separately collected five-minute request traces disabled strict
providers and therefore diagnose the generic fallback only:

• Ruby: initialize 9.909 s; all 150 documentSymbol requests completed in
  0.437 s total; 63 reference requests started, 47 finished, and 16 remained in
  flight at cutoff. Completed reference requests had already accumulated
  3,599.502 s of server time, with one taking 104.281 s.
• Dart: initialize 0.030 s; all 324 documentSymbol requests completed in
  0.485 s total; 1,204 reference requests started, 1,188 finished, and 16
  remained in flight. Completed requests accumulated 533.410 s of server time,
  with one taking 249.253 s.

Do not cite those traces as proof that scip-ruby or scip-dart alone takes 30
minutes. They prove that the fallback's declaration-proportional reference
fan-out is pathological.

Why the generic LSP route has a structural ceiling

scanSession.ts
requests document symbols per file and references per declaration. The
LSP 3.17 specification
defines editor-position requests, not a whole-program semantic database export.
Increasing concurrency merely transfers a declaration-sized queue into the
server and increases memory, cancellation, and mixed-generation pressure.

SCIP removes request fan-out but has a different ceiling. Its common adapter can
only use fields a producer actually writes, and producers disagree about roles,
relationship direction, containment, diagnostics, kinds, and identity. On
current master the registered facts are:

Provider	Authority	Fact families current master can claim
ttscgraph	compiler	exports, calls, accesses, instantiates, type refs, extends, implements, overrides, renders
samchon-graph-go	compiler	contains, exports, imports, calls, accesses, instantiates, type refs, implements, dispatches, tests, references
rust-analyzer-scip	semantic index	contains, references
scip-clang	semantic index	none under the strict common grounding rules
scip-java (Java/Kotlin)	semantic index	contains, references
scip-dotnet	semantic index	none under the strict common grounding rules
scip-python	semantic index	references
scip-ruby	semantic index	none under the strict common grounding rules
scip-php	semantic index	none under the strict common grounding rules
scip-dart	semantic index	none under the strict common grounding rules
samchon-graph-lua	analyzer	references

This table describes what the graph can defend, not every field present in an
artifact. Empty facts from a provider and a complete proof that a fact is absent
are not the same statement.

Decision: Graph Snapshot Protocol

Add a versioned Graph Snapshot Protocol above the existing bulk-session
transaction. It is a normalized producer contract, not another query API.
NDJSON is sufficient for the first implementation; switch to a
length-delimited binary encoding only after profiling shows serialization is a
material share.

One transaction consists of:

1. hello: protocol/schema version, provider and producer identity,
   compiler/analyzer version, languages, authority, supported fact families and
   optional capabilities;
2. begin: generation, optional base generation, exact build-universe digest,
   ordered source/config/dependency manifest digest, and target/configuration
   coordinates;
3. upsertShard: a content-addressed source or target shard containing nodes,
   edges, diagnostics, coverage and unresolved records;
4. deleteShard: an explicit removal for deleted/excluded inputs; and
5. commit: ordered shard manifest and fact digest.

Cancellation, producer failure, unknown schema, duplicate identity, digest
mismatch, source movement, or universe movement before commit leaves the last
published graph untouched. IBulkGraphSession.current remains a complete
immutable snapshot; sessions may apply deltas internally and expose the
reconstructed full snapshot only after validation. This avoids forcing every
consumer to understand partial states.

Shard identity and invalidation

A shard key includes:

• provider protocol and producer/compiler version;
• language, build target/source set and build configuration;
• source/checker content digest;
• dependency ABI/import-resolution fingerprint;
• generated-source/macro/annotation-processing inputs; and
• every language-specific semantic coordinate: Cargo features/cfg/target, C/C++
  compile command, JVM classpath and compiler flags, Scala BSP target, Swift
  triple/unit set, Dart package configuration, Python execution environment,
  and equivalents.

Never key only by mtime. Never share shards between incompatible targets.
Public-API/config/dependency changes may invalidate a dependent closure; the
producer must report which shards moved and why. A body-only edit should reuse
unaffected shards. A no-op should validate/load the prior manifest without
running a compiler.

The fastest implementation is inside or immediately beside the resident
semantic database. It must reuse the native compiler/analyzer's incremental
state. It must not:

• issue one relationship request per declaration;
• run clean;
• disable the build daemon, configuration cache, or incremental compiler;
• start a second cold compiler to enrich an index produced by the first; or
• reparse every source after a semantic index already parsed it.

Build-integrated plugins may write changed shards during the ordinary compile.
When the build reports UP-TO-DATE, the graph loads the validated existing
manifest without invoking the plugin.

Completeness and uncertainty contract

Static analysis cannot make reflection, runtime metaprogramming, dynamic import
paths, or unexecuted virtual dispatch exact. “Complete MCP information” means
publishing every determined fact and making every limit machine-readable, not
inventing runtime certainty.

For every (provider, language, target, GraphEdgeKind) publish exactly one
coverage row:

State	Contract
complete	Within the named universe, absence is meaningful.
partial	Proven facts are present, but unresolved sites or excluded inputs remain.
unsupported	This producer cannot prove the family.

Add an unresolved record with source evidence, target/fact kind, provider,
build target, stable reason and optional compiler-proven candidates. The closed
reason union initially contains:

dynamic, reflection, macro-or-generated, conditional-build,
external-boundary, analysis-error, excluded-input, identity-unstable,
and provider-gap.

Do not encode a possible runtime target as an executed dispatches edge.
Where the compiler proves only a candidate set, keep the candidates on the
unresolved record. Emit dispatches only when the selected universe proves the
runtime target is exact, such as a sealed/final hierarchy with one legal
implementation.

Framework-derived renders, tests, and some decorates facts use a named,
versioned enricher and their own coverage row. They must remain distinguishable
from language/compiler facts.

Preserve the contract through MCP

The dump already has provider capabilities, but
SamchonGraphMemory
does not retain dump.provenance, and
ISamchonGraphApplication.IOutput
returns only audit, next, and result. Its prose currently overclaims that
the graph holds “every symbol, call, type, decorator and test.”

Required consequence surface:

1. add normalized coverage and unresolved to the snapshot and dump;
2. validate full 15-family coverage for every strict slice;
3. retain provenance/coverage/unresolved in SamchonGraphMemory;
4. return an operation-scoped structured coverage summary and unresolved
   summary beside audit; and
5. update the MCP/JSDoc trust language so a caller can distinguish
   fact validation from fact-family completeness.

The request union and operation semantics remain stable. The output addition
must be a coordinated, versioned application-contract revision (or an additive
optional capability during migration), with the parity fixture updated in the
same change. Do not hide this information only in an English audit string.

Language-owned final routes

Issue	Final semantic owner and export route
#72 Rust	pinned rust-analyzer extension over one immutable Analysis/HIR revision
#73 C/C++	thin clangd fork retaining full Clang index roles, locals and relations in background-index shards
#74 Java	public javac plugin after ANALYZE, integrated with ordinary Gradle/Maven incremental compilation; JDT workspace route for resident IDE state
#75 C#	resident Roslyn MSBuildWorkspace service over immutable Solution, Compilation, SemanticModel and IOperation
#76 Kotlin	K2 compiler plugin integrated through KGP incremental tasks; pinned Analysis API service only where resident edit latency requires it
#77 Scala	Scala 2/3 typed compiler plugins driven by BSP/Zinc; SemanticDB as the declaration/relationship cross-check
#78 Swift	SourceKit-LSP/IndexStoreDB hook freezing an explicit compiler unit set, plus one source enrichment pass
#79 Zig	versioned exporter added at Zig compiler Sema/Zcu/InternPool, connected to the incremental build-server/watch path
#80 Python	pinned current Pyright Program and type-evaluator exporter, retaining Any/Unknown receiver coverage
#81 Ruby	Rubydex/RubyIndexer structural plane plus Sorbet typed overlay and sigil/untyped coverage
#82 PHP	PHPStan Collector over Node + Scope, using parallel workers and result cache
#83 Lua	pinned LuaLS custom command over the resident workspace VM, building a reverse fact table once per changed workspace
#84 Dart	pinned Analysis Server custom request over its resident AnalysisDriver/AnalysisSession

SCIP and generic LSP remain truthful compatibility fallbacks. They do not close
their child issue unless that issue's semantic and lifecycle gates pass.

Performance acceptance

These are targets to verify on pinned corpora, not claims about current master:

• true cold, empty native and graph caches: zero declaration-proportional RPCs;
  graph time no more than 1.25 × the measured native project
  load/typecheck/index baseline plus graph serialization;
• empty graph cache with valid native semantic shards: load/export only, measured
  separately from true cold;
• resident no-op: p95 below 250 ms, or no producer invocation when a validated
  disk snapshot is available;
• private body edit: p95 below 2 s on the pinned language fixture and work
  proportional to the native analyzer's invalidated closure;
• public signature/config/dependency edit: dependent invalidation is allowed,
  but every invalidated shard and cause is traceable and no hidden clean
  occurs;
• memory: no unbounded retention of obsolete generations and no duplicate
  compiler model solely for graph enrichment; and
• semantic quality: exact fixture assertions for identities, nodes, all 15 edge
  families, diagnostics, coverage and unresolved records. Non-zero counts are
  insufficient.

The benchmark matrix must separate project preparation, native semantic work,
export, normalization/merge and MCP readiness for:

true-cold, native-warm/graph-cold, resident-no-op, body-edit,
public-API-edit, create, delete, rename, configuration-change,
dependency-change, analysis-error, cancel, crash, and retry.

Do not rerun or publish a paid agent benchmark as part of this issue without its
separate authorization.

Implementation order

1. Land the common coverage/unresolved schema, canonical shard manifest, delta
   application, and MCP preservation with a deterministic fake producer.
2. Convert one already-complete resident provider (Go or TypeScript) to the
   protocol as the reference implementation and prove no-op/edit behavior.
3. Implement child providers independently against the frozen protocol. Each
   child owns its upstream pin, build-universe coordinates, identity rules,
   exact fact matrix and real-language fixture.
4. Update #143 only from the registry and verified provider metadata after a
   child route ships.
5. Keep #159's repository-topology plane generation-compatible but independent;
   it must not block language-semantic providers.

Done when

• The normalized protocol and shard store survive malformed frames, stale
  bases, cancellation, crash, retry, delete and configuration changes
  without losing the prior generation.
• Every published strict slice has exhaustive 15-family coverage and
  structured unresolved records.
• Provenance, coverage and unresolved summaries reach every MCP response.
• The MCP trust prose no longer claims unsupported completeness.
• No final strict provider performs per-symbol relationship fan-out or a
  second cold semantic pass.
• No-op, edit and cold gates above are measured against native baselines
  with phase-separated traces.
• Every #72-#84 issue passes its identity, semantic, lifecycle, platform and
  real-corpus gates.
• Generic LSP and graph-sitter remain available and truthfully downgraded.
• Focused tests, pnpm build, pnpm test, and pnpm coverage pass.

Primary references

• LSP 3.17 specification
• rust-analyzer architecture
• gopls implementation
• Clang indexing roles
• Kotlin Analysis API
• SemanticDB specification
• IndexStoreDB
• Pyright
• PHPStan Collectors

Start with IGraphProvider, IBulkGraphSession, BatchGraphSession, and the resident project transaction to understand the existing lifecycle and snapshot guarantees. Then read scanSession.ts and the strict checks under tests/experiment before tracing the SamchonGraphMemory MCP response path. Done means a versioned producer protocol supports validated shard updates, completeness and unresolved records, incremental reuse, and preserves the contract through MCP without replacing the last valid snapshot.