Add support for uploading entire directories to Azure blob storage
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 304
- Forks
- 45
- PR merge metrics
- No merged PRs in 30d
Description
Currently, MC2 Client doesn't support uploading entire directories to Azure blob storage. We'd like to add support for this, as data encrypted in sql format is always outputted as a directory with a data sub-directory and a schema sub-directory.
To do this, we'll have to investigate how to upload/download directories to/from Azure blob storage using the Azure Python SDK, and modify the upload_data() and download_data() functions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with python-package/mc2client/toolchain/encrypt_and_upload.py, focusing on upload_data() and download_data(). Review the Azure Python SDK documentation for directory-style blob uploads and downloads, including the data and schema sub-directories produced by sql-format encryption. Done means both functions support transferring those directory contents to and from Azure Blob Storage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, python
- Domain
- cloud, data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100