Shopify / Shopify/packwerk

[Feature Request] Dependency graph introspection

Open
#230 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Ruby
Stars
1.9k
Forks
121
Avg merge
2m
Merged PRs (30d)
1

Description

Hi, we have a use-case for packwerk that could significantly cut down on CI work needed for monolithic apps.
If we could tell which pack a changed file in a merge request belongs to and which packs depend on the changed pack,
it would be possible to run CI for only the affected part of the code, not necessarily all domain.
For example, changes in front-end code would not necessitate running DB querier specs, because there's no
way queriers were affected.

Let's consider this example app:

app/
  domain/
    thing.rb
    package.yml
  io/
    thing_reader.rb
    package.yml
package.yml

The root pack lists both app/domain and app/io as dependencies, because we need both for the app to function.
app/domain has no dependencies, it is the core of our business. app/io depends on app/domain - contains front-end stuff etc.

Given a list of changed files in a merge request, we would love to be able to ask packwerk two things:

  1. What package(s) are the files in?
  2. What packages, followed all the way to dependency tree branch ends, are downstream of changed packages?

So for example if we have an MR that changes app/domain/thing.rb:

  1. What package(s) are the files in? - ['app/domain']
  2. What packages are downstream? - ['.', 'app/io']

Whereas if app/io/thing_reader.rb gets changed:

  1. What package(s) are the files in? - ['app/io']
  2. What packages are downstream? - ['.'] # root pack will always be downstream

Perhaps to answer the question "which pack specs need to be ran?" merging both of these lists is preferable,
so here's a proposed API:

# implicitly runs on root and in 'self only' mode - lists all packs in the project
$ packwerk packs
'.'
'app/domain'
'app/io'

$ packwerk packs "app/io/thing_reader.rb"
'app/io'

$ packwerk packs --dependents "app/io/thing_reader.rb"
'.'

$ packwerk packs --affected "app/io/thing_reader.rb"
'.'
'app/io'

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining packwerk's existing command-line interface and the package.yml examples in the issue. Define how packwerk should map changed files to packs, calculate downstream dependents, and expose the proposed packs, --dependents, and --affected queries; the example outputs describe the expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.