get-convex / get-convex/dryad

Break files into functions or classes

Open
#5 0 comments 0 reactions 0 assignees View on GitHub
difficult enhancement good first issue
Dominant language
TypeScript
Stars
40
Forks
7
PR merge metrics
No merged PRs in 30d

Description

Currently, the token limit of the gpt models prevents us from indexing large source files. If we broke the files up into smaller chunks (like classes, or even functions), we could index (nearly?) everything. Ideally we keep the largest chunks possible though so we don't lose semantic information about cross-method or cross-function calling patterns.

This could either be done with some language-specific parsing library... or even better, using some LLM-assisted chunking that can work for (nearly) any kind of source file. ( https://learn.microsoft.com/en-us/azure/search/vector-search-how-to-chunk-documents )

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the repository's current source indexing flow and the chunking requirements described in issue #5, then compare the language-specific and LLM-assisted approaches, including the linked Azure guidance. Done means large source files can be indexed within model token limits while keeping chunks as large as possible without losing cross-function or cross-method context.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
ai, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.