apache / apache/buildstream

Declaration order of dependencies affects staging order and cache keys

Open
#1,995 1 comment 1 reaction 0 assignees View on GitHub
bug
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

The intent of sorting dependencies (done in `loadelement.pyx`) was that the staging order would be independent of the declaration order of dependencies in `.bst` files. As part of the investigation into #1988, it has become clear that the comparison function is broken as it does not impose a total ordering. Besides resulting in different sort orders with different Python versions (addressed in #1993), this also means that reordering dependencies in `.bst` files sometimes results in a change in staging order, which in turn results in a change in cache key.

BuildStream should implement a better sorting algorithm that is independent of the declaration order. As this would change staging order and cache keys of existing projects, this should be opt-in per project.

Contributor guide

Open the contributing guide

Research direction

Start in loadelement.pyx and review the dependency comparison described in the investigation of #1988 and the Python-version issue in #1993. Define a sorting approach whose result does not depend on dependency declaration order, then determine how the behavior can be enabled per project. Done means reordered dependencies produce the same staging order and cache key without changing existing projects by default.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.