microsoft / microsoft/AzureStorageExplorer
Scenario: Make it easy to store large library of media files in archive blobs
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 452
- Forks
- 92
- Avg merge
- 15h 20m
- Merged PRs (30d)
- 3
Description
Is your feature request related to a problem? Please describe.
I have a large library of media files - personal photos and videos - which I would like to archive in the cloud.
This library contains tens of thousands files organized in folders and its size if few hundreds GBs
Currently I am using Amazon Glacier but I would like to move my cloud archive to Azure because archive blob prices are way better. To do this I need few features in Azure Storage Explorer that makes it easy for me to maintain low cost cloud archive for my media library:
- Upload a directory to storage container and move all files to archive tier in most cost effective way
- Synchronize storage container with archived blobs with my local directory (in case some of files have been changed, added or removed in local directory)
- Compute difference between storage container and local directory and generate a repot of differences
- Perform sync - upload new archived blobs, replace changed ones, delete removed ones - in most cost effective way
- Download selected files, subdirectories or directory in most cost effective way
The goal is to automate user workflow to a few simple operations that will allow use to treat Azure storage container as Low Cost cloud archive for user files. All blob tier management for individual files should be automated - with tens of thousands files organized in multiple directories it is prohibitive to do tier management operations at each blob level.
Describe the solution you'd like
***"Archive a folder" feature.
User selects a folder for arhival and selects Azure storage account as target and enters storage container name for this operation. Local folder maps to a container in storage account. Azure Storage Explorer creates a storage container and starts uploading files (to hot access tier). When upload for a file has finished Azure Storage Explorer sets blob tier to Archive automatically. User is only charged for hot tier for short duration files are there during upload. If non-empty storage container exists "Archive a folder" feature shows error and suggests to use "Synchronize storage container with local folder" feature. This will be helpful when upload of large number of files was terminated in the middle
*** "Synchronize archived storage container with local folder" feature
User selects storage container and local folder for synchronization.
Azure Storage Explorer enumerates files in local folder tree and retrieves archived blob metadata from storage container.
Azure Storage Explorer computes difference between two sets of files: archived blobs and local files and generates actionable report for files that are different in local folder: (a) new files; (b) changed files; (c) deleted files. User can select what types of modifications should be synchronized. For (c) early deletion warning should be displayed if files are removed before 180 days.
Sync operation should also warn if some of the files in storage container are in "non-archive" tier.
All tier manipulation also happens automatically during Sync operation. Resulting blobs should be all set to Archive tier when operation completes.
*** "Full or partial download of archived files" feature
User selects blobs to be downloaded and starts download. Azure Storage Explorer automatically sends rehydration request for all archived files and informs the user to resume download when blobs are available in online tier. User should have an option to set blobs back to Archive tier after download.
*** "Archive all blobs in a container or specific blobs with common path prefix". This operation can be used to bulk archive blobs in storage container once user decides to move them all to Archive tier. This feature can be used to set blobs back to Archive tier after using "Full or partial download of archived files" feature.
Describe alternatives you've considered
I am using Amazon Glacier and Amazon FastGlacier application (ProVersion) developed by third-party. I believe Azure Storage Explorer should have comparable competitive features to make low cost archival accessible for wide user audience and businesses too. Current features where blob tier management happens on individual blob level are too prohibitive to use for large number of files.
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by reviewing the existing individual blob tier-management, upload, download, and rehydration flows described in the issue. Done would require a defined, tested scope for archive-folder uploads, synchronization, downloads, and bulk tier operations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 18/100