elastic / elastic/semantic-code-search-indexer

bug: TSX files fail to parse (typescript grammar used for .tsx)

Open
#137 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
TypeScript
Stars
19
Forks
10
PR merge metrics
No merged PRs in 30d

Description

## Problem
The indexer currently configures TypeScript parsing with `tree-sitter-typescript`'s **TypeScript** grammar, but registers the language for both `.ts` and `.tsx`. When a `.tsx` file is parsed with the TS grammar, the parse contains `ERROR` nodes and common queries (e.g. `lexical_declaration`) don’t match, which can yield **0 chunks** for TSX files.

## Why this matters
- TSX files can silently produce **no indexed chunks**, which makes them undiscoverable by search.
- This is a correctness gap independent of aggregation.

## Reproduction (no Elasticsearch required)
From the repo root:

1) Install deps
```bash
npm ci
```

2) Run this Node repro:
```bash
node -e 'const Parser = require("tree-sitter"); const TS = require("tree-sitter-typescript");
const src = "export const X = () =>

Hello
\\n";
const queryStr = "(lexical_declaration) @x";
function analyze(lang){
const p = new Parser();
p.setLanguage(lang);
const tree = p.parse(src);
const rootStr = tree.rootNode.toString();
const q = new Parser.Query(lang, queryStr);
const matches = q.matches(tree.rootNode).length;
return { hasError: rootStr.includes("ERROR"), matches };
}
console.log("TS.typescript:", analyze(TS.typescript));
console.log("TS.tsx:", analyze(TS.tsx));'
```

### Expected
- `TS.tsx.hasError` is `false`
- `TS.tsx.matches` is `>= 1`

### Actual (current behavior)
- `TS.typescript.hasError` is `true`
- `TS.typescript.matches` is `0`

## Where in code
- `src/languages/typescript.ts` currently sets `parser: ts.typescript` while also including `.tsx` in `fileSuffixes`.

## Suggested fix
- Use `tree-sitter-typescript`’s `tsx` grammar for `.tsx` files.

Implementation options:
1) **Split configs**: keep `typescript` for `.ts` with `ts.typescript`, add a new config (e.g. `tsx`) for `.tsx` with `ts.tsx`, reusing the same queries/importQueries/symbolQueries/exportQueries.
2) **Suffix-aware parser**: extend `LanguageConfiguration` to allow `parser` to be a map keyed by suffix, and select parser based on file extension.

## Test plan
- Add a unit test that feeds a minimal TSX file (containing a JSX element) into `LanguageParser.parseFile(...)` and asserts `chunks.length > 0`.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with src/languages/typescript.ts and inspect how LanguageParser.parseFile(...) selects the parser for file suffixes. Run the provided Node reproduction after npm ci, then add the planned unit test with a minimal TSX file containing JSX. Done means TSX parsing produces no grammar errors and chunks.length is greater than zero.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.