wolfi-dev / wolfi-dev/wolfictl

Remaining perf work for `wolfictl text`

Open
#250 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement needs-triage
Dominant language
Go
Stars
73
Forks
86
Avg merge
7d 7h
Merged PRs (30d)
15

Description

Description

I'm going to stop digging into this, but I wanted to write down what I've found before dropping all context:

Once all the PRs I've sent are merged and those dependencies get bumped, wolfictl text ~400ms instead of ~250s (on my machine, at least).

What's left looks like this:

image

The biggest chunk in the middle is calling BuildFlavor, from here:
https://github.com/chainguard-dev/melange/blob/a3b7a002e874b75c318e5d2a2c3c7af7142f456c/pkg/build/pipeline.go#L97-L98

We end up Stating (I think) an empty directory a lot, so fixing that would shave off ~50ms.

There's also a ton of redundant reading of files and parsing of yamls and detecting of commits. I added some logging to see what's going on:

   7 loadUse.ReadFile("pipelines/cmake/install.yaml")
   8 loadUse.ReadFile("pipelines/cmake/build.yaml")
   8 loadUse.ReadFile("pipelines/cmake/configure.yaml")
  15 loadUse.ReadFile("pipelines/go/install.yaml")
  17 loadUse.ReadFile("pipelines/meson/configure.yaml")
  19 loadUse.ReadFile("pipelines/meson/compile.yaml")
  20 loadUse.ReadFile("pipelines/meson/install.yaml")
  31 loadUse.ReadFile("pipelines/ruby/clean.yaml")
  32 loadUse.ReadFile("pipelines/ruby/build.yaml")
  32 loadUse.ReadFile("pipelines/ruby/install.yaml")
  45 loadUse.ReadFile("pipelines/go/build.yaml")
 113 loadUse.ReadFile("pipelines/patch.yaml")
 166 loadUse.ReadFile("pipelines/autoconf/configure.yaml")
 185 loadUse.ReadFile("pipelines/git-checkout.yaml")
 253 loadUse.ReadFile("pipelines/autoconf/make-install.yaml")
 262 loadUse.ReadFile("pipelines/autoconf/make.yaml")
 495 loadUse.ReadFile("pipelines/strip.yaml")
 526 loadUse.ReadFile("pipelines/fetch.yaml")
 687 detectCommit(".")
4576 Stat("lib/libc.so.6")
4576 Stat("lib64/libc.so.6")

I'm a little hesitant to start refactoring things too much, but it seems like it would be straightforward to cache this stuff.

There are also a bunch of little things:

We spend a ton of time in https://github.com/chainguard-dev/go-apk/blob/2829525a71369b8c570b98332bb73639b1e59802/pkg/apk/version.go#L371 doing regex stuff. If it's possible to do this in a non-regexy way, we could save some time.

All of the string manipulation in here could also be a lot faster: https://gitlab.alpinelinux.org/alpine/go/-/blob/master/repository/repository.go

E.g. IndexUri(), NewRepositoryFromComponents, and Url should all be using path.Join instead of fmt.Sprintf.

The Packages() method should initialize pkgs with Count() capacity.

I suspect we could get this down to sub-100ms if we fix everything.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by profiling wolfictl text and reviewing the BuildFlavor call referenced in the issue, along with the logged repeated ReadFile, detectCommit, and Stat operations. Identify which caching or parsing changes are safe, then measure the result; the stated goal is to reduce runtime toward sub-100ms without changing command behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.