Determine the maximum item count limit for bulk operations
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
Bulk operations (bulk_\*) process items individually with partial-failure support. Without an upper bound on the number of items per request, a single bulk call can cause excessive DB load, long-running transactions, and degraded responsiveness.
This task is to determine an appropriate maximum item count limit for bulk operations through testing and validation.
Scope:
- Benchmark bulk operations (create/upsert/update/purge) at increasing item counts to find performance and stability thresholds.
- Identify the practical maximum item count per bulk request based on measured DB load, latency, and transaction behavior.
- Recommend a concrete limit value to enforce at the API/service layer.
JIRA Issue: BA-6281
Contributor guide
Research direction
Start by locating the bulk_* API/service handlers and their create, upsert, update, and purge paths. Benchmark increasing request sizes while recording DB load, latency, and transaction behavior. Done means identifying a measured stability threshold and documenting a concrete maximum item count for enforcement at the API/service layer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100