microsoft / microsoft/lage

[RFC]: "link" command for accelerating remote cache setup

Open
#374 0 comments 0 reactions 1 assignee View on GitHub

@kenotron is already working on this.

Since Sep 22, 2022.

dev design
Dominant language
TypeScript
Stars
816
Forks
88
PR merge metrics
No merged PRs in 30d

Description

Overview

Currently, there is no command that one can use to set up remote cache. The built-in remote cache provider is "Azure Blob Storage". However, it is through backfill's documentation (readme) that users get a HINT on how to set up remote cache. The documentation also does not cover how a developer of a repo would be able to access the remote cache as read-only blobs.

Goals & non-goals

Goals
  1. give users a single command to
  • sign in to azure cli
  • create or choose a blob container
  • save the R/O connection string in lage.config.js
  • give users a command to retrieve & display the correct environment variables for ADO pipelines
  1. document this command in the docs site
  • in the tutorial section (?)
  • in the cookbooks section as "Remote Cache with Azure Blob Storage"
Upstream dependencies
  1. backfill
  2. azure-cli potentially

Test Plan

  • using a mocked azure cli interface, we should create snapshots of expected outputs of the hint messages
  • unit tests should be employed to make sure the connection string generation is correct

Performance, Resilience, Monitoring

  • we need to validate the version of the upstream dep of azure-cli before executing it

Security & Privacy

  • we should NEVER store the readonly blob storage SAS token - the expiry should be set to a relatively short period so as to make sure we can ensure it will not expire during a build, but short enough that the compromised token would not last long
  • read-write tokens are to be generated MANUALLY (access keys to blob container) and stored as part of the standard pipeline secrets mechanism (keyvault, etc)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.