BONSAMURAIS / BONSAMURAIS/vacuum_pump
Design document on where to store data backups / data to be imported
- Dominant language
- Shell
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Most data is available, but we probably don't want to be hitting these external sites too often. At the same point, hosting large files ourselves costs money. Also, it could be the case that we process a lot of raw data and store the results in the DB - what do we do with the raw data, if anything?
Clear guidelines and best practices are needed here. Doesn't need to be perfect the first time around, this isn't the final version!
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named. Start by reviewing how the project currently obtains external data, imports it, stores processed results in the relational database, and handles raw files. Done means a first-pass design document with clear guidance for backups, external-source access, hosted files, and raw-data retention.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, shell
- Domain
- data-engineering, databases, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100