[RFC]: "link" command for accelerating remote cache setup
Open
@kenotron is already working on this.
Since Sep 22, 2022.
dev design
- Dominant language
- TypeScript
- Stars
- 816
- Forks
- 88
- PR merge metrics
- No merged PRs in 30d
Description
Overview
Currently, there is no command that one can use to set up remote cache. The built-in remote cache provider is "Azure Blob Storage". However, it is through backfill's documentation (readme) that users get a HINT on how to set up remote cache. The documentation also does not cover how a developer of a repo would be able to access the remote cache as read-only blobs.
Goals & non-goals
Goals
- give users a single command to
- sign in to azure cli
- create or choose a blob container
- save the R/O connection string in lage.config.js
- give users a command to retrieve & display the correct environment variables for ADO pipelines
- document this command in the docs site
- in the tutorial section (?)
- in the cookbooks section as "Remote Cache with Azure Blob Storage"
Upstream dependencies
backfillazure-clipotentially
Test Plan
- using a mocked azure cli interface, we should create snapshots of expected outputs of the hint messages
- unit tests should be employed to make sure the connection string generation is correct
Performance, Resilience, Monitoring
- we need to validate the version of the upstream dep of
azure-clibefore executing it
Security & Privacy
- we should NEVER store the readonly blob storage SAS token - the expiry should be set to a relatively short period so as to make sure we can ensure it will not expire during a build, but short enough that the compromised token would not last long
- read-write tokens are to be generated MANUALLY (access keys to blob container) and stored as part of the standard pipeline secrets mechanism (keyvault, etc)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.