chroma-core / chroma-core/chroma

[Bug]: Socket failed to close properly

Open
#2,702 0 comments 0 reactions 1 assignee Claimed by @itaismith View on GitHub
bug
Dominant language
Rust
Stars
29.3k
Forks
2.5k
Avg merge
1d 4h
Merged PRs (30d)
38

Description

### What happened?

I am running Chroma in client/server mode and accessing it using Python. I noticed that after the result is returned, the connection does not terminate properly and remains in the CLOSE_WAIT state.

- tcp 0 0 0.0.0.0:8848 0.0.0.0:* LISTEN 724516/python
- tcp 1 0 127.0.0.1:42802 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:35690 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:45512 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:52992 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:38088 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:43532 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:39846 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:34114 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:42214 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:60566 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:44490 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:38070 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:35902 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:48292 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:55546 127.0.0.1:8848 CLOSE_WAIT 783538/python
- tcp 1 0 127.0.0.1:45360 127.0.0.1:8848 CLOSE_WAIT 783538/python
I have a large number of such connections in my system.
Here is my code:
```
import uuid

import chromadb

from ..document import HandleDocument
from ..embedding import OllamaEmbedding
from ..entity import AddDto, DeleteDto, SelectDto

class ChromaManager:
def __init__(self, host: str = 'localhost', port: int = 8848):
self.chroma_client = chromadb.HttpClient(host=host, port=port)

def reset(self):
return self.chroma_client.reset()

def get_collection(self):
return self.chroma_client.list_collections()

def add_collection(self, name: str, model_name: str):
return self.chroma_client.create_collection(
name=name,
embedding_function=OllamaEmbedding(model_name=model_name)
)

def delete_collection(self, name: str):
self.chroma_client.delete_collection(name)

def _get_collection(self, name: str, model_name: str):
return self.chroma_client.get_collection(
name=name,
embedding_function=OllamaEmbedding(model_name=model_name)
)

def add_documents(self, dto: AddDto):
doc_type_methods = {
("xls", "xlsx"): lambda: HandleDocument(dto.url, dto.chunk_size, dto.chunk_overlap).excel(),
("pdf",): lambda: HandleDocument(dto.url, dto.chunk_size, dto.chunk_overlap).pdf(),
("ppt", "pptx"): lambda: HandleDocument(dto.url, dto.chunk_size, dto.chunk_overlap).ppt(),
("doc", "docx"): lambda: HandleDocument(dto.url, dto.chunk_size, dto.chunk_overlap).docx(),
("html",): lambda: HandleDocument(dto.url, dto.chunk_size, dto.chunk_overlap).html(),
("csv",): lambda: HandleDocument(dto.url, dto.chunk_size, dto.chunk_overlap).csv(),
}

method = None
for types, func in doc_type_methods.items():
if dto.type in types:
method = func
break

if method is None:
raise ValueError("Unsupported document type")

splits = method()

chroma_collection = self._get_collection(name=dto.name, model_name=dto.model)

ids = []
metadata = []
documents = []

for doc in splits:
ids.append(str(uuid.uuid1()))
metadata.append(doc.metadata)
documents.append(doc.page_content)

chroma_collection.add(
ids=ids,
metadatas=metadata,
documents=documents
)

return {"num_splits": len(splits), "ids": ids}

def select_document(self, dto: SelectDto):
chroma_collection = self._get_collection(name=dto.name, model_name=dto.model)
data = chroma_collection.get(
where={"source": dto.url},
offset=dto.offset,
limit=dto.limit
)
return data

def delete_document(self, dto: DeleteDto):
chroma_collection = self._get_collection(name=dto.name, model_name=dto.model)
chroma_collection.delete(
where={"source": dto.url}
)

```

### Versions

Chroma:0.5.5
Python:3.11.9
os:20.04.1-Ubuntu

### Relevant log output

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.