elastic / elastic/semantic-code-search-indexer
bug: TSX files fail to parse (typescript grammar used for .tsx)
- Dominant language
- TypeScript
- Stars
- 19
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
The indexer currently configures TypeScript parsing with `tree-sitter-typescript`'s **TypeScript** grammar, but registers the language for both `.ts` and `.tsx`. When a `.tsx` file is parsed with the TS grammar, the parse contains `ERROR` nodes and common queries (e.g. `lexical_declaration`) don’t match, which can yield **0 chunks** for TSX files.
## Why this matters
- TSX files can silently produce **no indexed chunks**, which makes them undiscoverable by search.
- This is a correctness gap independent of aggregation.
## Reproduction (no Elasticsearch required)
From the repo root:
1) Install deps
```bash
npm ci
```
2) Run this Node repro:
```bash
node -e 'const Parser = require("tree-sitter"); const TS = require("tree-sitter-typescript");
const src = "export const X = () =>
const queryStr = "(lexical_declaration) @x";
function analyze(lang){
const p = new Parser();
p.setLanguage(lang);
const tree = p.parse(src);
const rootStr = tree.rootNode.toString();
const q = new Parser.Query(lang, queryStr);
const matches = q.matches(tree.rootNode).length;
return { hasError: rootStr.includes("ERROR"), matches };
}
console.log("TS.typescript:", analyze(TS.typescript));
console.log("TS.tsx:", analyze(TS.tsx));'
```
### Expected
- `TS.tsx.hasError` is `false`
- `TS.tsx.matches` is `>= 1`
### Actual (current behavior)
- `TS.typescript.hasError` is `true`
- `TS.typescript.matches` is `0`
## Where in code
- `src/languages/typescript.ts` currently sets `parser: ts.typescript` while also including `.tsx` in `fileSuffixes`.
## Suggested fix
- Use `tree-sitter-typescript`’s `tsx` grammar for `.tsx` files.
Implementation options:
1) **Split configs**: keep `typescript` for `.ts` with `ts.typescript`, add a new config (e.g. `tsx`) for `.tsx` with `ts.tsx`, reusing the same queries/importQueries/symbolQueries/exportQueries.
2) **Suffix-aware parser**: extend `LanguageConfiguration` to allow `parser` to be a map keyed by suffix, and select parser based on file extension.
## Test plan
- Add a unit test that feeds a minimal TSX file (containing a JSX element) into `LanguageParser.parseFile(...)` and asserts `chunks.length > 0`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with src/languages/typescript.ts and inspect how LanguageParser.parseFile(...) selects the parser for file suffixes. Run the provided Node reproduction after npm ci, then add the planned unit test with a minimal TSX file containing JSX. Done means TSX parsing produces no grammar errors and chunks.length is greater than zero.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100