EmbeddingFunction lacks async support, blocks event loop in AsyncTable.add()
- Dominant language
- Rust
- Stars
- 11.4k
- Forks
- 1.1k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 140
Description
## Problem
`EmbeddingFunction` methods (`compute_source_embeddings`, `compute_query_embeddings`) are synchronous-only. When used with `AsyncTable`, this blocks the event loop in production async frameworks like FastAPI.
### Inconsistency between `add()` and `search()`
`AsyncTable.search()` correctly wraps embedding calls in `run_in_executor`:
```python
# table.py – AsyncTable.search()
async def make_embedding(embedding, query):
loop = asyncio.get_running_loop()
return (await loop.run_in_executor(
None, embedding.function.compute_query_embeddings_with_retry, query,
))[0]
```
But `AsyncTable.add()` calls embeddings **synchronously on the event loop** via `_append_vector_columns()`:
```python
# table.py – _append_vector_columns()
col_data = func.compute_source_embeddings_with_retry(batch[conf.source_column])
```
This blocks all concurrent requests when ingesting data with registered embedding functions.
### Broader issue
The `EmbeddingFunction` ABC has no async interface at all. The TypeScript SDK already supports `async computeSourceEmbeddings()` / `async computeQueryEmbeddings()`, but the Python SDK has no equivalent. Most embedding providers (OpenAI, Cohere, Bedrock, etc.) offer async clients that could be leveraged.
## Suggested improvements
1. **Short-term**: Wrap `compute_source_embeddings_with_retry` in `run_in_executor` inside `AsyncTable.add()`, matching the pattern already used in `AsyncTable.search()`
2. **Long-term**: Add `async` variants to `EmbeddingFunction` (`async_compute_source_embeddings`, `async_compute_query_embeddings`) so implementations can use native async HTTP clients instead of thread pool workarounds
## Environment
- lancedb 0.30.2
- Python 3.13
- FastAPI
Contributor guide
Research direction
Start in table.py with AsyncTable.add(), _append_vector_columns(), and the existing AsyncTable.search() run_in_executor path. Then inspect the Python EmbeddingFunction ABC and compare its methods with the TypeScript SDK's async methods. Confirm whether the intended scope is the short-term executor fix, the long-term async interface, or both, and verify that async ingestion no longer blocks the event loop.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fastapi, python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100