algolia / algolia/algoliasearch-client-python
feat: Edit the requests configuration pool_maxsize variable when initializing client
- 主要语言
- Python
- 星标
- 207
- 派生
- 63
- PR 合并指标
- 30 天内没有已合并 PR
描述
- Algolia Client Version: 2.4.0
- Language Version: Python 3.9.1
### Description
I'm replicating a dataset (60k records) with 100 threads (Python's builtin _ThreadExecutor_) at a time. And I find that from time to time I keep getting these warnings from the 'requests' library: **"Connection pool is full, discarding connection: xxxxxxxx.algolia.net"**.
The ability to edit the _pool_maxsize_ configuration variable when initializing the search client would solve this kind of issues, perhaps even allowing to edit other _requests_ configurations.
### Steps To Reproduce
Initialize a ThreadExecutor of 100 max workers.
Create a function that uses 'save_object' to store a record in a predefined index.
Loop through thousands of records, for each record:
-Add the function that saves the record with the record data to the executor
You should get the warning every ~1000 records.
贡献指南
调研方向
从搜索客户端初始化路径开始,跟踪其 requests 会话或连接池是如何配置的。验证并发的 save_object 调用如何使用该配置;完成的标准是调用方在创建客户端时可以设置 pool_maxsize,并且 100 个 worker 的复现不再发出连接池已满的警告。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- api, backend
- Issue 类型
- 功能
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 25/100