chroma-core / chroma-core/chroma

[ENH] Create custom serialization format for middleware

Open
#2,844 0 comments 0 reactions 0 assignees View on GitHub
by-chroma
Dominant language
Rust
Stars
29.3k
Forks
2.5k
Avg merge
1d 4h
Merged PRs (30d)
38

Description

FastAPI doesn't support the serialization of numpy arrays, which forces us to convert embeddings to and from python lists. Check out `_add` in `chromadb/api/fastapi.py`, for example:

```
batch = (
ids,
convert_np_embeddings_to_list(embeddings),
metadatas,
documents,
uris,
)
```

And then on the other side of things, `add` in `chromadb/server/fastapi/__init__.py`:

```
return self._api._add(
collection_id=_uuid(collection_id),
ids=add.ids,
embeddings=cast(
Embeddings,
[np.array(embedding) for embedding in add.embeddings]
if add.embeddings
else None,
),
metadatas=add.metadatas, # type: ignore
documents=add.documents, # type: ignore
uris=add.uris, # type: ignore
)
```

A custom serialization format not reliant on FastAPI would get rid of this issue and allow us to improve memory performance by not having to convert back and forth.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading `_add` in `chromadb/api/fastapi.py` and `add` in `chromadb/server/fastapi/__init__.py`, tracing where embeddings are converted to lists and back to NumPy arrays. Define a middleware serialization approach that removes those conversions and verify that embedding data still reaches the API correctly with improved memory behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, numpy, python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.