microsoft / microsoft/typeagent-py
Streaming commit transaction does embedding and fuzzy index work that could be pipelined
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 884
- Forks
- 75
- Avg merge
- 3h 2m
- Merged PRs (30d)
- 2
Description
_commit_batch_streaming (conversation_base.py line 434) opens a transaction and calls _update_secondary_indexes_incremental inside it. That function does two expensive things:
-
Embedding generation —
_update_message_index_incremental→message_index.add_messages()→text_location_index.add_text_locations()→_embedding_index.add_texts(). These are API calls to the embedding model, happening inside the DB transaction. -
Fuzzy index term embeddings —
_update_related_terms_incrementalcollects new terms from semantic refs and callsfuzzy_index.add_terms(), which generates an embedding per unique term. Many terms repeat across batches; theCachingEmbeddingModelhelps but the per-batch overhead of collecting and checking is still there.
The two-stage pipeline already overlaps LLM extraction(N+1) with commit(N). But the commit phase itself is slower than necessary because of these embedding calls. Pre-computing embeddings alongside knowledge extraction (before the transaction opens) would keep the commit phase to pure DB writes. The fuzzy index could also be deferred to a single pass after all batches complete, since it's only needed for query-time — not for correctness of the ingested data.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in conversation_base.py at _commit_batch_streaming (line 434), then trace _update_secondary_indexes_incremental through _update_message_index_incremental and _update_related_terms_incremental. Separate embedding work from the transaction and defer fuzzy index updates until all batches complete; done means commit performs only database writes while ingested data remains correct and query-time fuzzy indexing is available afterward.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data, search
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100