IntelLabs / IntelLabs/vdms

Local Storage Naming and Directory Scaling

Open
#188 0 comments 0 reactions 0 assignees View on GitHub
Discussion Needed Enhancement Priority: Backlog
Dominant language
C++
Stars
93
Forks
34
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
Currently there is a static directory hierarchy that's constructed for local storage of videos, images, etc, on server startup.

While a fan-out is necessary to prevent overloading a single storage location (i.e. directory), we should (re)investigate this and consider refactoring as its somewhat inefficient and rigid for larger scale use-cases.

We also have a somewhat naive method for generating local path and file names that may have collision risks at larger scale that are not well articulated.

**Describe the solution you'd like**

Ideally we should have a methodology that is akin to consistent hashing in distributed storage. This lets us dynamically scale and adjust our hierarchy.

As for internal naming, we could look at UUIDs to be automatically generated.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.