EpistasisLab / EpistasisLab/tpot
New feature: MongoDB or other NOSQL for memory
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
The "memory" in TPOT caches candidate pipeline metadata on disk.
The data is cached as a large number of very small ASCII text files each containing data represented by a JSON object.
It occurs to me that this is a crude "document database", and both time and space efficiencies could be gained by using something like MongoDB instead.
I have also discovered my "Real Time Protection" anti-malware software is scanning every file in the cache. This has a devastating impact on performance!
Recommend that users with any anti-virus software installed should configure anti-malware settings to white-list (exclude) the location of the joblib memory cache for performance.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.