SOLR error: exceeded limit of maxWarmingSearchers=2
- Dominant language
- Python
- Stars
- 212
- Forks
- 81
- PR merge metrics
- No merged PRs in 30d
Description
When I load a large number of datasets with multiple processes using ckanapi like this:
``` ckanapi load datasets -I /data/ckan.jsonl.gz -z -p 4 -c /etc/ckan/testing/development.ini ```
Eventually, a Solr error is thrown:
``` ERROR [ckan.model.modification] Solr returned an error: (u'Solr responded with an error (HTTP 503): [Reason: Error opening new searcher. exceeded limit of maxWarmingSearchers=2, try again later.]',) ```
I could increase the number of WarmingSearchers in Solr or decrease the number of workers for the load, but IMHO the bigger issue here is whether it could be possible for bulk loading to have an option to commit to Solr only after loading all datasets, not one by one, as it still takes a long time this way (9 hours for 130k datasets to load using single worker)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.