algolia / algolia/algoliasearch-client-python

feat: Edit the requests configuration pool_maxsize variable when initializing client

未关闭
#521 1 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
207
派生
63
PR 合并指标
30 天内没有已合并 PR

描述

- Algolia Client Version: 2.4.0
- Language Version: Python 3.9.1

### Description
I'm replicating a dataset (60k records) with 100 threads (Python's builtin _ThreadExecutor_) at a time. And I find that from time to time I keep getting these warnings from the 'requests' library: **"Connection pool is full, discarding connection: xxxxxxxx.algolia.net"**.

The ability to edit the _pool_maxsize_ configuration variable when initializing the search client would solve this kind of issues, perhaps even allowing to edit other _requests_ configurations.

### Steps To Reproduce
Initialize a ThreadExecutor of 100 max workers.
Create a function that uses 'save_object' to store a record in a predefined index.
Loop through thousands of records, for each record:
-Add the function that saves the record with the record data to the executor

You should get the warning every ~1000 records.

贡献指南

打开贡献指南

调研方向

从搜索客户端初始化路径开始,跟踪其 requests 会话或连接池是如何配置的。验证并发的 save_object 调用如何使用该配置;完成的标准是调用方在创建客户端时可以设置 pool_maxsize,并且 100 个 worker 的复现不再发出连接池已满的警告。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
api, backend
Issue 类型
功能
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。