samchon / samchon/compiler-knowledge-graph
[Bulk index][Rust] Export rust-analyzer HIR snapshots beyond SCIP
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 6
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Outcome
Add a pinned rust-analyzer graph exporter that walks one immutable HIR/IDE
revision and emits complete or explicitly bounded Rust semantic shards without
textDocument/references fan-out. Keep the already-shipped
rust-analyzer scip provider as a navigation fallback; it is not the final
provider and does not close this issue.
Current master baseline
PR #147 ships
rustScipProvider.
It requires rust-analyzer, rustc, Cargo and the scip decoder; invokes
rust-analyzer scip . --exclude-vendored-libraries; and publishes
semantic-index authority with only contains and references.
That limitation is proven in upstream source, not inferred from a sparse
fixture. Stock rust-analyzer's SCIP writer hardcodes an empty relationship list
and empty diagnostics and only marks the definition role. It cannot express
trait implementations, overrides, resolved calls, constructions, access roles,
test relationships or dispatch. Parsing formatted SCIP monikers to recover
those facts is forbidden: the rendered trait/self path is not a stable
compiler identity.
The exact PR #147 Tokio cell took 55.238 s; the strict-off rust-analyzer LSP
cell took 229.861 s. Both are cold end-to-end results. No no-op or edit
latency has been measured.
Upstream feasibility result
The final route is feasible but is not a stable crates.io sidecar.
Rust-analyzer's architecture
keeps a fully resolved incremental semantic model, applies changes through
AnalysisHost, and hands readers immutable Analysis snapshots. HIR and IDE
queries are backed by Salsa and therefore already have the invalidation behavior
the graph needs.
The API boundary is the cost:
cli::sciplives in the rust-analyzer binary crate;- workspace crates such as
hir,ide, andide_dbare not a versioned
public crates.io SDK; and - Cargo project loading executes build scripts and proc macros and is sensitive
to target, cfg, features and toolchain.
Therefore choose, in order:
- an upstreamed versioned
graphSnapshotcommand; - a git-pinned wrapper crate built at an exact rust-analyzer commit; or
- a narrow maintained fork containing only the exporter and compatibility
handshake.
A plain external parser or stock SCIP post-processor is not an acceptable final
route.
Exporter design
Add samchon/graphSnapshot or an equivalent CLI/NDJSON command in the pinned
rust-analyzer tree. For each request:
- capture one immutable
Analysisplus the exact project-model/Cargo revision; - walk crates/modules and source-owned HIR declarations once;
- resolve expression, type and relationship facts through HIR/IDE queries;
- serialize changed source/crate shards using #63's Graph Snapshot Protocol;
- re-check the revision and Cargo universe; and
- commit only if both remain unchanged.
The traversal must cover:
- crates, modules, visibility and re-exports;
- structs, enums, unions, traits, impls, functions, methods, associated items,
constants, statics, fields, variants, type aliases, generics and locals; - resolved function/method calls, constructors, field reads/writes and type
references; - direct trait implementations, supertraits and override/implementation
relationships; - attributes and test functions/modules;
- macro and proc-macro declarations/uses with expansion-to-source evidence; and
- diagnostics from the same revision.
Do not retain rust-analyzer HIR handles across revisions. Project graph nodes
are serialized values; the next refresh reacquires semantic handles from the
new immutable Analysis.
Identity and universe
Use rust-analyzer definition identities internally, then publish a versioned
stable identity containing the Cargo package identity, crate disambiguator,
module/owner path, item kind and structural signature where overload-like
generated items require it. Do not use display-formatted monikers as the sole
key.
Explicit cases:
- inherent and trait methods with the same name stay distinct;
- an impl method identifies both its owning impl and implemented trait item;
local N-style document IDs never escape without a file/owner namespace;- macro-generated twins retain expansion/call-site provenance;
- cfg-gated twins are separated by the build universe, not silently merged; and
- generic monomorphizations do not become false source declarations.
The universe digest includes Cargo metadata, workspace/package graph, lockfile,
features (including defaults), target triple, cfg set, profile, rustc/Cargo and
rust-analyzer versions, build-script outputs, proc-macro enablement and relevant
environment/configuration. The stock SCIP artifact does not record most of
these; the provider must capture them from the project model and toolchain.
Completeness rules
Publish all 15 graph fact-family coverage rows.
- Direct calls and inherent dispatch may be
completefor the selected
universe. - Trait-object and generic dispatch is
partialunless rust-analyzer proves a
closed candidate set. Candidates belong on an unresolved record, not as
executeddispatches. - Code excluded by cfg/features is
conditional-build, not proven absent. - Disabled/failed build scripts or proc macros are
macro-or-generated/analysis-error. - External dependency bodies excluded by policy are
external-boundary. rendersis unsupported unless a separately named framework enricher is
added.
Macro-expanded facts must never masquerade as handwritten spans. Store
call-site and definition-site evidence when rust-analyzer can map both.
Incremental and performance design
Keep the rust-analyzer process resident and use its normal change path. Let
Salsa determine the affected query closure. Persist graph shards independently
from rust-analyzer's memory so process restart can validate and load an
unchanged generation.
- unchanged revision: return the prior manifest/digest without walking every
crate; - body edit: export only changed/invalidated owner/source shards;
- public item/Cargo feature/target change: invalidate the compiler-reported
dependent closure and report it; - deletion/rename: emit explicit shard deletions; and
- crash/cancellation/revision movement: retain the last committed graph.
Cold acceptance is relative to rust-analyzer project load plus cache priming:
no declaration-proportional RPCs and no second compiler analysis. On the pinned
fixture, target resident no-op p95 below 250 ms and body-edit p95 below 2 s.
Implementation map in this repository
- Extend the common shard/coverage contract from #63.
- Add a new provider beside
provider/rust,
leavingrustScipProvideras fallback. - Add resolver/configuration logic that pins rust-analyzer tag and commit
and verifies rustc/Cargo compatibility before selection. - Add protocol parsing and identity/coverage validation; reject unknown
handshake versions. - Add a deterministic fake-session fixture before the real upstream binary.
- Extend
tests/experimentwith the HIR provider as the preferred row and
retain a stock-SCIP comparison row. - Update the registry/docs only after the real provider passes.
Acceptance
- No strict-HIR path invokes
textDocument/referencesper declaration. - Exact fixtures cover workspaces, re-exports, features, target cfg,
inherent/UFCS/trait calls, same-named trait methods, impls, supertraits,
associated items, generics, closures, async, macros, proc macros, build
output and tests. - Source and macro evidence is attributable to one immutable revision.
- All 15 fact families have
complete,partial, orunsupported
coverage and unresolved reasons. - Unchanged, body edit, public API edit, create, delete, rename, Cargo
metadata, feature/target, macro failure, cancel, crash and retry are
atomic. - Cold time is compared with native rust-analyzer load/prime-caches; no-op
and edit targets above are measured separately. - The provider reports exact rust-analyzer commit, rustc, Cargo, target,
features and cfg universe. - Stock SCIP and generic LSP remain truthful fallbacks.
- Focused tests, Rust experiment,
pnpm build,pnpm test, and
pnpm coveragepass.
Non-goals
- inferring trait relations from SCIP strings;
- treating potential dynamic implementations as executed targets;
- indexing every cfg/target universe in one graph; or
- depending on
scip-rust, which is only a wrapper around the built-in
rust-analyzer command.
Primary references
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with packages/graph/src/provider/rust/rustScipProvider.ts and the common shard/coverage contract from #63, then read rust-analyzer's cli::scip entry point and architecture reference. Add the pinned HIR provider beside the existing Rust provider, use the deterministic fake-session fixture first, and run the focused tests plus tests/experiment; done requires the listed coverage, identity, incremental, atomicity, and performance checks to pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, typescript
- Domain
- compilers, devtools, testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100