lancedb / lancedb/lancedb

EmbeddingFunction lacks async support, blocks event loop in AsyncTable.add()

Open
#3,268 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
11.4k
Forks
1.1k
Avg merge
1d 15h
Merged PRs (30d)
140

Description

## Problem

`EmbeddingFunction` methods (`compute_source_embeddings`, `compute_query_embeddings`) are synchronous-only. When used with `AsyncTable`, this blocks the event loop in production async frameworks like FastAPI.

### Inconsistency between `add()` and `search()`

`AsyncTable.search()` correctly wraps embedding calls in `run_in_executor`:

```python
# table.py – AsyncTable.search()
async def make_embedding(embedding, query):
loop = asyncio.get_running_loop()
return (await loop.run_in_executor(
None, embedding.function.compute_query_embeddings_with_retry, query,
))[0]
```

But `AsyncTable.add()` calls embeddings **synchronously on the event loop** via `_append_vector_columns()`:

```python
# table.py – _append_vector_columns()
col_data = func.compute_source_embeddings_with_retry(batch[conf.source_column])
```

This blocks all concurrent requests when ingesting data with registered embedding functions.

### Broader issue

The `EmbeddingFunction` ABC has no async interface at all. The TypeScript SDK already supports `async computeSourceEmbeddings()` / `async computeQueryEmbeddings()`, but the Python SDK has no equivalent. Most embedding providers (OpenAI, Cohere, Bedrock, etc.) offer async clients that could be leveraged.

## Suggested improvements

1. **Short-term**: Wrap `compute_source_embeddings_with_retry` in `run_in_executor` inside `AsyncTable.add()`, matching the pattern already used in `AsyncTable.search()`
2. **Long-term**: Add `async` variants to `EmbeddingFunction` (`async_compute_source_embeddings`, `async_compute_query_embeddings`) so implementations can use native async HTTP clients instead of thread pool workarounds

## Environment

- lancedb 0.30.2
- Python 3.13
- FastAPI

Contributor guide

Open the contributing guide

Research direction

Start in table.py with AsyncTable.add(), _append_vector_columns(), and the existing AsyncTable.search() run_in_executor path. Then inspect the Python EmbeddingFunction ABC and compare its methods with the TypeScript SDK's async methods. Confirm whether the intended scope is the short-term executor fix, the long-term async interface, or both, and verify that async ingestion no longer blocks the event loop.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.