chroma-core / chroma-core/chroma

[Bug]: Sanitize OpenAI input

Open
#1,503 4 comments 0 reactions 0 assignees View on GitHub
bug by-chroma
Dominant language
Rust
Stars
29.3k
Forks
2.5k
Avg merge
1d 4h
Merged PRs (30d)
38

Description

### What happened?

If texts passed to OpenAI API are breaking the JSON our wrapper sends 400 error is returned. Maybe there is a way we can sanitize output or leave it to openai lib to fix that.

https://discord.com/channels/1073293645303795742/1183798062863372318

### Versions

Any

### Relevant log output

```shell
ERROR:root:Error: Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}
Traceback (most recent call last):
File "/home/matischroder/sanatorio_allende/upload_web_chroma.py", line 94, in
main()
File "/home/matischroder/sanatorio_allende/upload_web_chroma.py", line 87, in main
collection_id = train_with_url(
File "/home/matischroder/sanatorio_allende/upload_web_chroma.py", line 77, in train_with_url
raise e
File "/home/matischroder/sanatorio_allende/upload_web_chroma.py", line 66, in train_with_url
chroma_collection.add(
File "/home/matischroder/sanatorio_allende/venv/lib/python3.9/site-packages/chromadb/api/models/Collection.py", line 147, in add
embeddings = self._embed(input=documents)
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure from the reported upload_web_chroma.py call chain, especially chromadb.api.models.Collection.add and its embedding path. Determine whether the wrapper or the OpenAI library should handle text that breaks the JSON request; done means affected inputs produce valid requests without the reported 400 error and the behavior is covered by a relevant test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.