PowerShell / PowerShell/DSC

Maintenance: Add caching for build

Open
#170 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
523
Forks
75
Avg merge
3d 16h
Merged PRs (30d)
24

Description

Summary of the new feature / enhancement

Right now, the builds take 15-20m on average. This seems primarily due to the compiling and building of the various projects. If some of those builds and downloads could be cached - or parallelized - we could probably get the build down to 5-10m.

Proposed technical implementation details (optional)

I think we have the following opportunities for improving build times:

  • Cache the downloads for external crates
  • Cache the builds for internal crates and only rebuild if their code changed
  • Cache the downloads for PowerShell modules

Possibly, we could:

  • Parallelize the builds for internal crates into groups that depend on each other:

    ---
    title: Dependency Map
    ---
    flowchart LR
        dsc_lib       --> dsc
        dsc_lib       --> test_group_resource
        ntstatuserror --> ntreg
        ntstatuserror --> ntuserinfo
        ntstatuserror --> registry
        ntreg         --> registry
        ntuserinfo    --> ntreg
        osinfo
        pal           --> registry
        process
        y2j
    
    ---
    title: Build Jobs
    ---
    flowchart LR
        osinfo
        process
        y2j
        dsc_lib          ---> dsc
        dsc_lib          ---> test_group_resource
        ntstatuserror     --> ntreg+ntuserinfo
        ntreg+ntuserinfo  --> registry
        pal              ---> registry
    
    • Build job for dsc_lib is a dependency for dsc and test_group_resource
    • Build job for ntstatuserror is a dependency for a job that builds both ntuserinfo and ntreg
    • Build job for pal and ntreg+ntuserinfo are dependencies for registry
    • osinfo, process, and y2j are independent builds
    • All build jobs are a dependency for acceptance tests

    While this sounds very complicated, in practice we can use GitHub artifacts to pass builds between jobs and reuse prior builds if the code didn't change. There's some prior art we can use for this. I'd be willing to prototype the workflow.

    That being said, I think we could definitely start with the initial caching and move on from there if the build times are still very high.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file or test is named. Start by locating the GitHub Actions build workflow and its Rust crate and PowerShell module build steps, then measure the current build; done means the proposed caching or parallelization works without breaking acceptance tests and reduces build time.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, powershell, rust
Domain
build-system, ci-cd, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.