ADORSYS-GIS / ADORSYS-GIS/lci-codegraph
[Ticket]: Chunk and graph symbol sets diverge on every tags language, and nothing tests that they agree
- Dominant language
- Rust
- Stars
- 0
- Forks
- 1
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 18
Description
## Summary
The two halves of a walk discover symbols by **two independent mechanisms**, and the sets they produce
disagree — in both directions.
- Chunks always come from `interesting_node` (`src/chunk.rs:169`), a single flat `match` on
tree-sitter node kinds shared by every language.
- The graph comes from that same matcher **only for Rust** (`GraphStrategy::RustNative`). Python,
JavaScript, TypeScript, TSX and Java go through the grammar's bundled `tags.scm`
(`GraphStrategy::Tags`).
`src/lang/mod.rs:28` states the Rust case explicitly — its own extractor exists "so chunk + graph
symbols stay in lock-step." That is accurate, and it is also an unflagged statement that for the tags
languages they are not.
Nothing in the test suite asserts any relationship between the two sets, so the drift is free to grow.
## Reproduction
Verified against `main` @ `293d12f`, `build_graph(true)`, default `IndexTuning`. Verbatim output.
**TypeScript** — three graph-only symbols, and one chunk that cannot be matched to its node:
```ts
export const arrowFn = (a: number): number => { return a + 1; };
export interface Repo {
find(id: string): T | null;
}
@sealed
export class Decorated { ... }
```
```
CHUNKS (5):
[function] name=None lines 0..2 <- this is arrowFn
[class] name=Some("Decorated") lines 9..22
[method] name=Some("load") lines 12..17
[method] name=Some("count") lines 19..21
[function] name=Some("sealed") lines 24..24
GRAPH (7):
api.ts#1:arrowFn api.ts#5:Repo api.ts#6:find
api.ts#10:Decorated api.ts#13:load api.ts#20:count api.ts#25:sealed
```
- `Repo` (interface) and `find` (its method) are graphed, never chunked —
`interface_declaration` has no arm in `interesting_node`.
- `arrowFn` **is** chunked, at the right line, but **anonymously**: `arrow_function` yields
`("function", None)` and `variable_declarator` is explicitly skipped as "too noisy", so the name
never reaches the chunk. The graph knows it as `arrowFn`.
**Python** — an anonymous, duplicated, *mistyped* chunk:
```py
@functools.total_ordering
class Ordered:
def __init__(self):
self.v = 0
```
```
CHUNKS:
[function] name=None lines 4..7 <- the decorated class
[class] name=Some("Ordered") lines 5..7
[function] name=Some("__init__") lines 6..7
GRAPH:
mod.py#6:Ordered mod.py#7:__init__
```
`decorated_definition => ("function", None)` fires regardless of whether the decorated thing is a
`def` or a `class`, so a decorated **class** is emitted as a chunk of type `"function"` with no name —
overlapping the correctly-typed `Ordered` chunk that the recursion then produces. Two embeddings of
nearly the same bytes, one of them mislabelled and unnamed.
## What this does and does not break
Worth being precise, because I expected worse and did not find it:
- **Line numbers agree.** Across every fixture I ran, wherever both passes found the same symbol, the
chunk's `start_line + 1` equalled the graph node's `start_line`. I found **no** off-by-one drift.
- **The failure is set membership and missing names.** Symbols present in one pass and absent from the
other, plus chunks carrying `symbol_name: None` where the graph has a real name.
So a host reconstructing a node id as `{file_path}#{start_line + 1}:{name}` gets the right answer
whenever both sides agree a named symbol exists — and silently gets nothing for `arrowFn`, `Repo`,
`find`, every Java method, and every decorated Python definition. It is an undocumented,
untested coupling between two passes that were never specified to agree.
## Why the test suite did not catch this
The golden (`tests/golden/sample-repo.graph.json`) snapshots the **graph only**. Chunk output is
snapshotted separately. No test compares the two sets, in either direction, for any language — so
every divergence above is invisible to `cargo test` and always has been.
## Suggested direction
Two independent pieces; the first is cheap and worth doing regardless.
1. **Close the specific gaps in `interesting_node`**: `interface_declaration` (TS/Java), and make
`decorated_definition` inspect its inner definition to inherit the right `chunk_type` and name
rather than hardcoding `("function", None)`. For `arrow_function` / `function_expression` bound to
a `variable_declarator`, lift the binding's name — this is exactly what `tags.scm` already does,
and it is why the graph has `arrowFn` and the chunk does not.
2. **Add a test that asserts the two sets relate**, per language. Even a weak invariant — every
*named* graph definition node has a chunk covering its start line — would have caught every case
in this issue and in the Java one. Without it, the next grammar bump moves one pass and not the
other, silently.
The deeper question — whether the two passes should share one symbol table rather than being
reconciled after the fact — is worth asking but is a larger change than this issue proposes.
## Related
- #10 is the same root cause and by far the worst instance; filed separately because its
impact (whole-class chunks, `max_chunk_lines` defeated) is qualitatively different.
- Distinct from #8, which is about `calls` edge resolution — a different pass entirely.
## Provenance
Found while designing an integration that consumes chunks and graph together and needs to join them.
Reproduced independently against `main` @ `293d12f` with a scratch binary; all output above is
verbatim from that run.
AI usage: drafted with Claude (Claude Code). Every claim was produced by executing the unmodified
crate. Note in particular the "line numbers agree" finding — that contradicts what I initially
predicted from reading the source, and is stated here because the run disagreed with me.
Contributor guide
Assessment
This issue has not been assessed yet.