VectorStore/VectorIndex methods should allow to specify multiply Sources
Open
Nobody has claimed this yet.
enhancement
good first issue
- Dominant language
- C#
- Stars
- 1.1k
- Forks
- 141
- Avg merge
- 2d 18m
- Merged PRs (30d)
- 2
Description
From this thread: https://discord.com/channels/1115206893015662663/1115206893544161302/1222715558181736539
Simplify code like this:
var source1 = await PdfPigPdfSource.CreateFromUriAsync(new Uri("https://canonburyprimaryschool.co.uk/wp-content/uploads/2016/01/Joanne-K.-Rowling-Harry-Potter-Book-1-Harry-Potter-and-the-Philosophers-Stone-EnglishOnlineClub.com_.pdf"));
var source2 = await PdfPigPdfSource.CreateFromUriAsync(new Uri("https://canonburyprimaryschool.co.uk/wp-content/uploads/2016/01/Joanne-K.-Rowling-Harry-Potter-Book-1-Harry-Potter-and-the-Philosophers-Stone-EnglishOnlineClub.com_.pdf"));
var documents1 = await source1.LoadAsync(cancellationToken).ConfigureAwait(false);
var documents2 = await source2.LoadAsync(cancellationToken).ConfigureAwait(false);
var documents = documents1.Concat(documents2).ToArray();
var options = new SQLIteVectorStoreOptions();
var vectorStore = new SQLiteVectorStore(options.Filename, options.TableName, embeddings);
var textSplitter = new CharacterTextSplitter();
var indexCreator = new VectorStoreIndexCreator(vectorStore, textSplitter);
var index = await indexCreator.FromDocumentsAsync(documents).ConfigureAwait(false);
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing VectorStoreIndexCreator.FromDocumentsAsync and the VectorStore/VectorIndex methods used by the sample. Compare their current input shape with the manual loading and concatenation shown; done when multiple sources can be passed without that manual combination while preserving existing indexing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, sqlite
- Domain
- ai, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100