microsoft / microsoft/typeagent-py

Streaming commit transaction does embedding and fuzzy index work that could be pipelined

Open
#270 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
884
Forks
75
Avg merge
3h 2m
Merged PRs (30d)
2

Description

_commit_batch_streaming (conversation_base.py line 434) opens a transaction and calls _update_secondary_indexes_incremental inside it. That function does two expensive things:

  1. Embedding generation_update_message_index_incrementalmessage_index.add_messages()text_location_index.add_text_locations()_embedding_index.add_texts(). These are API calls to the embedding model, happening inside the DB transaction.

  2. Fuzzy index term embeddings_update_related_terms_incremental collects new terms from semantic refs and calls fuzzy_index.add_terms(), which generates an embedding per unique term. Many terms repeat across batches; the CachingEmbeddingModel helps but the per-batch overhead of collecting and checking is still there.

The two-stage pipeline already overlaps LLM extraction(N+1) with commit(N). But the commit phase itself is slower than necessary because of these embedding calls. Pre-computing embeddings alongside knowledge extraction (before the transaction opens) would keep the commit phase to pure DB writes. The fuzzy index could also be deferred to a single pass after all batches complete, since it's only needed for query-time — not for correctness of the ingested data.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in conversation_base.py at _commit_batch_streaming (line 434), then trace _update_secondary_indexes_incremental through _update_message_index_incremental and _update_related_terms_incremental. Separate embedding work from the transaction and defer fuzzy index updates until all batches complete; done means commit performs only database writes while ingested data remains correct and query-time fuzzy indexing is available afterward.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data, search
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.