samchon / samchon/compiler-knowledge-graph
[Bulk index][Java] Export compiler and JDT workspace snapshots without query fanout
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 6
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Outcome
Export Java graph shards from the compiler invocation that already type-checks
the project, using the public javac plugin/tree/model APIs after attribution.
Preserve Gradle/Maven incremental state and use a JDT workspace route only when
the resident IDE model is the semantic owner. Do not run clean, query JDT once
per declaration, or type-check the same source a second time for enrichment.
Current master baseline
Current master registers scip-java for Java and Kotlin. For Java it publishes
semantic-index authority with contains and references. The PR #147 Gson
cold cell took 88.653 s, compared with 231.398 s through generic JDT LS
fan-out.
scip-java 0.13.x changed its architecture:
- the project moved to
scip-code/scip-java; - javac/kotlinc plugins emit SCIP directly rather than SemanticDB;
- the supported floor is Java 17 and Gradle 8;
- Scala/Zinc support was removed; and
- the default Gradle/Maven path performs a real build and includes
clean.
Its artifact remains a useful navigation fallback, but SCIP has no call role.
Imports/accesses/implementation relationships are producer-specific, and the
order-based overload/constructor discriminator churns when declarations are
inserted or reordered.
Upstream mechanism
The public JDK compiler API is enough for the build-integrated route:
Plugin
is loaded byServiceLoaderon the processor path and receives the active
JavacTask;- a plugin registers a
TaskListener
for parse/enter/analyze/generate/compilation events; and Trees
maps attributedTreePaths toElement,TypeMirror, source positions,
scopes and declaration trees.
Elements/Types provide direct supertypes, member lookup and override checks.
No private javac AST parser is required for the initial strict provider.
Annotation-processor/generated source handling still needs lifecycle tests.
Decision: two semantic-owner lanes, one protocol
Build lane: javac plugin
Attach a versioned graph plugin to the project's ordinary Java compile tasks.
On ANALYZE completion for a compilation unit, scan its attributed tree once
and write a source shard. At final compilation completion, write the ordered
manifest/commit record only if diagnostics and the build universe satisfy the
configured policy.
Integrate through Gradle/Maven without replacing their task graph:
- Gradle: add the plugin/processor path and
-Xplugin:option to each actual
Java source-set compile task; retain daemon, build cache, configuration cache
and incremental compilation; - Maven: attach to the existing compiler execution and target directory; do not
prependclean; and - Bazel/other javac hosts: attach at the toolchain/action boundary only when the
exact target/action inputs are nameable.
If a task is UP-TO-DATE, load the previously committed shards. Deleted source,
classpath, compiler-option, annotation-processor or generated-source changes
must invalidate the affected manifest even when no plugin invocation occurs.
Resident lane: JDT
When an Eclipse/JDT workspace already owns unsaved/resident source, expose a
narrow bulk snapshot from that workspace and its reconciled compilation units.
It must produce the same graph schema and identities as the javac route and
must freeze one workspace generation. It is a fallback/IDE lane, not a second
pass after javac.
Fact extraction
Walk attributed trees and elements:
- declarations, visibility, nesting, packages/modules, imports/exports;
MethodInvocationTreeand member references to resolved executable elements;NewClassTree/constructor elements for instantiation;- identifiers/member selects plus assignment/update context for reads/writes;
TypeMirrorand tree paths for type references;- direct class/interface supertypes and
Elements.overridesfor inheritance and
overrides; - annotations with resolved annotation type and annotated target;
- diagnostics from the same compilation task; and
- test facts only through a named JUnit/TestNG enricher with framework
provenance.
Virtual/interface calls identify the statically selected method. Runtime
implementations remain candidate/unresolved unless final/sealed analysis proves
one exact target.
Identity and universe
Use module/artifact coordinate plus binary owner name, item kind and a canonical
structural signature:
- methods/constructors include erased or fully qualified parameter types,
generic arity and return/type-parameter context as required; - fields, record components, enum constants and annotation elements remain
distinct; - locals include source file, enclosing stable member and lexical structure;
- anonymous/local classes include a structural owner key that does not change
on unrelated line insertion; and - source/bridge/generated elements retain origin and generator/processor
provenance.
Do not trust scip-java's +N overload ordinal as the final ID. Exact fixtures
must insert/reorder overloads and constructors without renaming unaffected
members.
The universe includes module/source set, Java release/preview flags, classpath
and module path order/content, compiler/JDK version, annotation processors and
options, generated sources, Gradle/Maven target coordinates and relevant build
properties. One multi-module build can have several target universes.
Incremental and correctness behavior
- no-op: validate existing manifest; compiler need not run;
- private body edit: ordinary incremental compile writes only affected source
shards; - public ABI edit: build tool/compiler invalidates dependents, and the graph
records the affected target/source closure; - annotation processor or classpath change: invalidate generated/consumer
shards according to the build; - compiler error: publish diagnostics and either a clearly partial generation
or retain the prior strict generation according to the common policy—never
silently combine old and new facts; - delete/rename: explicit shard deletion; and
- cancellation/crash: no commit.
Cold acceptance is relative to the same build without the graph plugin. Target
overhead is at most 25% plus serialization, with no extra compilation.
Implementation map
- Implement the common shard and coverage schema from #63.
- Create a separately versioned Java exporter artifact and Gradle/Maven fixture
integration; do not put a JVM inside the TypeScript package. - Add a provider beside the existing scip-java entry and make it preferred
only after a handshake proves target coverage. - Normalize javac/JDT identities through one conformance fixture.
- Retain scip-java as a compatibility/navigation fallback.
- Add phase-separated real Gson measurement and lifecycle rows.
Acceptance
- One compile traversal emits declarations, calls, construction,
accesses, types, direct inheritance, overrides, annotations, diagnostics
and source evidence. - No
clean, no disabled Gradle daemon/configuration cache, and no second
compiler pass. - Multi-module Gradle and Maven fixtures cover main/test source sets,
JPMS, records, sealed types, overloads, constructors, generics, lambdas,
anonymous/local classes, annotation processing and generated sources. - Overload insertion/reorder and multiple constructors keep stable IDs.
- All 15 fact families have explicit coverage/unresolved rows.
- No-op, body/API edit, create/delete/rename, classpath/config/processor
change, error, cancel, crash and retry are atomic. - JDT and javac lanes agree on shared source facts and report different
producer/universe provenance. - Cold overhead is measured against the native compile; warm no-op and edit
latency are reported separately. - scip-java and generic JDT LS remain truthful fallbacks.
- Focused tests, Java experiment,
pnpm build,pnpm test, and
pnpm coveragepass.
Primary references
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the common shard and coverage schema from #63 and the existing scip-java provider entry. Read the public javac Plugin, TaskListener, and Trees APIs, then examine the required Gradle/Maven fixture integration and run the focused tests, pnpm build, pnpm test, and pnpm coverage. Done means the javac and JDT lanes satisfy the listed fact, identity, incremental, atomicity, fallback, and performance acceptance checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, typescript
- Domain
- build-system, compilers, devtools, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100