Huge CPU usage of qdrant client when deserializing payload

Title: High CPU Usage with Qdrant Async Client for GRPC and REST Endpoints

Description:

We are experiencing significant CPU usage spikes when enabling the Qdrant async client for both GRPC and REST endpoints. By utilizing a profiling tool (Parca), we have identified that a considerable portion of CPU time is being consumed by payload deserialization.

This issue is particularly problematic for applications like recommendation engines that rely on Qdrant as a vector database, as it results in a substantial increase in CPU requirements for our application.

Steps to Reproduce:

1. Enable Qdrant async client for both GRPC and REST endpoints.
2. Run the application and perform standard operations.
3. Monitor CPU usage and analyze with a profiling tool (e.g., Parca).

Expected Behavior:

• Efficient CPU usage when deserializing payloads.

Actual Behavior:

• High CPU usage is observed, primarily due to payload deserialization.

Environment:

• Qdrant version: [Specify version]
• GRPC and REST async client setup
• Profiling tool: Parca

Proposed Solution:

To mitigate the high CPU usage, we propose utilizing orjson for JSON serialization and deserialization. orjson is known for its performance benefits compared to the standard JSON library in Python. By replacing the current deserialization process with orjson, we anticipate a reduction in CPU overhead.

Additional Context:
This CPU overhead is posing a challenge in deploying Qdrant in resource-constrained environments. Any guidance or fixes to address this issue would be greatly appreciated.

Start by profiling the Qdrant async client's GRPC and REST payload deserialization with Parca, then identify the current JSON serialization path. Compare it with the proposed orjson approach and verify that standard operations show lower CPU usage without changing deserialized results.