aboutcode-org / aboutcode-org/vulnerablecode
Run importers in Parallel
Open
- Dominant language
- Python
- Stars
- 702
- Forks
- 328
- Avg merge
- 3d 8h
- Merged PRs (30d)
- 3
Description
Instead of running `./manage.py import --all` in sequence
We can use multiprocessing and try to use all CPU power.
suggestions:
- we can use Celery and schedule tasks and run easily importers in parallel
(another benefit: we can run periodic Tasks and make importers synchronize )
- we can make it simple and use some code from scancode.
this could be a solution for #833
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.