qdrant / qdrant/qdrant-client

serialization error while upserting

Open
#302 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.4k
Forks
304
Avg merge
2d 9h
Merged PRs (30d)
37

Description

Hi,
Im getting the following error.

Exception while upserting Error serializing to JSON: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcb0' in position 128: surrogates not allowed
Traceback (most recent call last):
File "dynamicbatch_v6.py", line 337, in writer
client.upsert(
File "/home/gl/huggingface/venv/lib/python3.8/site-packages/qdrant_client/qdrant_client.py", line 757, in upsert
return self._client.upsert(
File "/home/gl/huggingface/venv/lib/python3.8/site-packages/qdrant_client/qdrant_remote.py", line 1123, in upsert
http_result = self.openapi_client.points_api.upsert_points(
File "/home/gl/huggingface/venv/lib/python3.8/site-packages/qdrant_client/http/api/points_api.py", line 1179, in upsert_points
return self._build_for_upsert_points(
File "/home/gl/huggingface/venv/lib/python3.8/site-packages/qdrant_client/http/api/points_api.py", line 601, in _build_for_upsert_points
body = jsonable_encoder(point_insert_operations)
File "/home/gl/huggingface/venv/lib/python3.8/site-packages/qdrant_client/http/api/points_api.py", line 22, in jsonable_encoder
return to_json(
File "/home/gl/huggingface/venv/lib/python3.8/site-packages/qdrant_client/_pydantic_compat.py", line 34, in to_json
return model.model_dump_json(*args, **kwargs)
File "/home/gl/huggingface/venv/lib/python3.8/site-packages/pydantic/main.py", line 353, in model_dump_json
return self.pydantic_serializer.to_json(
pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcb0' in position 128: surrogates not allowed

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the failing client.upsert call in dynamicbatch_v6.py and trace the serialization path through qdrant_client/qdrant_remote.py, points_api.py, and _pydantic_compat.py. Reproduce the UnicodeEncodeError involving the surrogate character, then establish whether the client should handle or reject this input and verify the resulting behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.