microsoft / microsoft/lage

Local cache not working in CI pipeline

Open
#695 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
816
Forks
88
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
Local cache is not working in my CI pipeline, even though I have skipLocalCache: false in my config.

To Reproduce
Steps to reproduce the behavior:

  1. Create a lage.config.js with these options:
{
  pipeline: {
    build: {
      dependsOn: ["^build"],
      inputs: ["./src/**/*.*", "package.json"],
      outputs: ["./lib", "./build"],
    },
    test: ["build"],
    clean: {
      cache: false,
    },
  },
  cacheOptions: {
    // defaults to true in CI systems
    skipLocalCache: false,
  },
}
  1. Create a CI pipeline that runs lage build && lage test in the same job/stage
  2. Observe how lage runs the build command twice.

Expected behavior
Lage should skip through the build command and only run test in the second invocation since the packages are already built.

Environment/Troubleshooting
Happens in an Azure DevOps pipeline.

envinfo
  System:
    OS: Linux 5.15 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
    CPU: (2) x64 Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
    Memory: 5.70 GB / 6.77 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 18.17.1 - /opt/hostedtoolcache/node/18.17.1/x64/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 9.6.7 - /opt/hostedtoolcache/node/18.17.1/x64/bin/npm
    pnpm: 8.7.0 - /opt/hostedtoolcache/node/18.17.1/x64/bin/pnpm
  Managers:
    Apt: 2.4.10 - /usr/bin/apt
    Cargo: 1.71.1 - ~/.cargo/bin/cargo
    Composer: 2.5.8 - /usr/bin/composer
    Gradle: 8.3 - /usr/bin/gradle
    Maven: 3.8.8 - /usr/bin/mvn
    pip3: 22.0.2 - /usr/bin/pip3
    RubyGems: 3.3.5 - /usr/bin/gem
  Utilities:
    Bazel: 6.3.2 - /usr/local/bin/bazel
    CMake: 3.27.3 - /usr/local/bin/cmake
    Make: 4.3 - /usr/bin/make
    GCC: 11.4.0 - /usr/bin/gcc
    Git: 2.41.0 - /usr/bin/git
    Clang: 14.0.0 - /usr/bin/clang
    Mercurial: 6.1.1 - /usr/bin/hg
    Subversion: 1.14.1 - /usr/bin/svn
    Curl: 7.81.0 - /usr/bin/curl
  Servers:
    Nginx: 1.18.0 - /usr/sbin/nginx
  Virtualization:
    Docker: 24.0.5 - /usr/bin/docker
  SDKs:
    Android SDK:
      API Levels: 27, 28, 29, 30, 31, 32, 33, 33, 33, 34
      Build Tools: 27.0.0, 27.0.1, 27.0.2, 27.0.3, 28.0.0, 28.0.1, 28.0.2, 28.0.3, 29.0.0, 29.0.1, 29.0.2, 29.0.3, 30.0.0, 30.0.1, 30.0.2, 30.0.3, 31.0.0, 32.0.0, 33.0.0, 33.0.1, 33.0.2, 34.0.0
      Android NDK: 25.2.9519653
  IDEs:
    Nano: 6.2 - /usr/bin/nano
    Vim: 8.2 - /usr/bin/vim
  Languages:
    Bash: 5.1.16 - /usr/bin/bash
    Go: 1.20.7 - /usr/bin/go
    Java: 11.0.20 - /usr/bin/javac
    Perl: 5.34.0 - /usr/bin/perl
    PHP: 8.1.2 - /usr/bin/php
    Python: 3.10.12 - /usr/bin/python
    Python3: 3.10.12 - /usr/bin/python3
    R: 4.3.1 - /usr/bin/R
    Ruby: 3.0.2 - /usr/bin/ruby
    Rust: 1.71.1 - /home/vsts/.cargo/bin/rustc
  Databases:
    MySQL: 0.22.04.1 - /usr/bin/mysql
    SQLite: 3.37.2 - /usr/bin/sqlite3
  Browsers:
    Chrome: 116.0.5845.96
    Chromium: 116.0.5845.0

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the lage.config.js cacheOptions and reproduce the issue by running lage build && lage test in an Azure DevOps job. Trace why the second invocation does not reuse the first build's local cache; done means the build is skipped on the second invocation while test still runs as expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, typescript
Domain
build-system, ci-cd
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.