aboutcode-org / aboutcode-org/scancode-workbench
Suggestion to speed up initial JSON import
Open
performance
- Dominant language
- TypeScript
- Stars
- 173
- Forks
- 81
- PR merge metrics
- No merged PRs in 30d
Description
### Description
When I import a JSON file, I noted that the sqlite DB file seems to be flushed on disk incrementally and I can see the sqlite file grow as the import goes.
I have an idea: if you import to a memory only database and once this is all imported and only then you save on disk this may speed things up? this may be worth a try
### System configuration
**Operating System**:
Linux Ubuntu 16.04 x86_64
**ScanCode Workbench Version**:
3.0.0b2
Contributor guide
Assessment
This issue has not been assessed yet.