livepeer / livepeer/go-livepeer
On-disk local storage
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 586
- Forks
- 226
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 19
Description
**Is your feature request related to a problem? Please describe.**
Currently, segments within in-memory local storage are rotated out after about a [dozen segments](https://github.com/livepeer/go-livepeer/blob/master/drivers/local.go#L12).
For various reasons, an external OS may not be available or desirable (cost, latency, centralization, ease of testing or integration).
On-disk local storage would meet our persistence requirements without external OS. Some features which may drive this requirement:
* Recording/DVR or time shifting of live streams
* VOD playback
* Caching segments locally for verification
* Faster local accesses: https://github.com/livepeer/go-livepeer/issues/846#issuecomment-489236016
* External OS latency
**Describe the solution you'd like**
On-disk local storage driver. There are a few ways to integrate this:
* Replace in-memory storage entirely
* Keep this separate from in-memory storage and select which to use at start-up time via CLI flag
* Hybrid approach (serve from in-memory if in cache, otherwise hit disk). Questionable whether this is actually significantly faster given the page cache is usually pretty good. Needs benchmarking (and benchmarking might entail spending time implementing it; catch-22)
* Hybrid approach based on usage. Eg, VOD/DVR served from disk while live streams are served from memory.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with drivers/local.go and trace how in-memory segments are rotated and accessed. Compare the issue's replacement, separate CLI-selected, and hybrid approaches, noting that the hybrid option requires benchmarking. Done means the project has a defined on-disk local storage integration that provides the requested persistence for recording, VOD, caching, or related use cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100