Optimise intchron()
- Dominant language
- R
- Stars
- 5
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
We should try to make IntChron queries as fast as possible. Currently, retrieving the entire database (without tabulation) takes:
```r
system.time(intchron("all", tabulate = FALSE))
#> user system elapsed
#> 48.587 0.579 436.211
```
`user`+`system` time is low (but could still be improved), so we can assume most of the time is spent waiting for responses to IntChron requests.
Ideas for optimisation:
- [ ] Group batches of requests into vectorised calls to `intchron_request()` as often as possible
- [ ] Parallelise requests:
- [ ] At the least it should be safe to execute recursive calls to `intchron_crawl()` in parallel, since crawling each branch is independent of other parts of the tree.
- [ ] Also investigate using ... in `intchron_request()` and possibly elsewhere.
- [ ] But need to ensure these don't place too much load on the IntChron server, since we already get timeouts when running the relatively modest queries in the examples and vignettes simultaneously as R CMD checks.
- [ ] Improve filtering/pattern-matching options in `intchron_crawl()` and use them aggressively in `intchron()`
- [ ] Investigate where the `user` CPU time is being spent
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.