concurrency settings

As you might imagine while our content has grown so too has the time to index. Is there anyway the concurrency settings of scrapy could be exposed? or some feature to run sections of an index in parallel be added to a roadmap?

e.g settings like this
```
REACTOR_THREADPOOL_MAXSIZE = 128
CONCURRENT_REQUESTS = 256
CONCURRENT_REQUESTS_PER_DOMAIN = 256
CONCURRENT_REQUESTS_PER_IP = 256
```

The issue identifies Scrapy concurrency settings and parallel indexing as possible directions, but does not name files, tests, or a chosen scope. First determine how this scraper starts indexing and loads Scrapy settings; done requires an agreed implementation path, exposed configuration or parallelism behavior, and tests demonstrating the resulting concurrency.

Written by the indexing model from the issue text.