kernelci / kernelci/kernelci-project
KernelCI mini-CDN (storage solution)
- Dominant language
- Python
- Stars
- 14
- Forks
- 32
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 3
Description
Right now we are using Azure files storage, which is not very good for storing artifacts by multiple reasons:
* Very slow API to upload, list, delete many files
* We have no control over storage on downloads, we cannot check if someone have improper caching, or if someone is downloading too much
* Generated URL is "ugly" and not very user friendly
* Generally cloud storage is very expensive
We need to investigate better storage solution, that is likely will implement function of "mini-CDN".
* Keep "hot" artifacts in fast storage (block device)
* Keep "cold" artifacts that might be used from Azure in Azure files mounted by NFS to "CDN" VM
* Keep "cold" artifacts that might be used by other users on some cheap storages - Hetzner, etc.
This subject need to be investigated, and probably idea is improved.
Another prior research: https://github.com/kernelci/kernelci-project/issues/310
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the prior research in issue #310, then compare Azure Files with the proposed block-device and lower-cost storage options. Investigate how hot and cold artifacts could be served through a mini-CDN, including the NFS-mounted Azure Files path. Done means documenting an improved storage design and recommendation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100