algorand / algorand/algorand-sdk-testing

CI: Prevent merging algorand-sdk-testing PRs that break SDKs

オープン
#165 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
Infrastructure new-feature-request Team Scytale
主要言語
Gherkin
スター
21
フォーク
35
平均マージ
15分
マージ済み PR(30日)
1

説明

## Problem

Merging algorand-sdk-testing in a state that breaks downstream SDKs in master _shouldn’t_ happen. But, in practice, it happens without mal-intent.

Background:
* Since there’s no build check in algorand-sdk-testing, it’s possible to unintentionally merge test changes that break SDKs.
* Since there’s no time-based SDK build process (e.g. nightly), no one discovers a broken SDK build until someone tries to build the project. It’s difficult to correlate a build failure with root problem cause.
* Motivating example:
* Unintentionally, https://github.com/algorand/algorand-sdk-testing/pull/162 was merged without corresponding JS and Python SDK changes.
* Post-merge, folks surfaced the problem through community contribution PRs (e.g. https://github.com/algorand/py-algorand-sdk/pull/271).
* After surfacing the problem, the gap was closed:
* https://github.com/algorand/js-algorand-sdk/pull/540/
* https://github.com/algorand/py-algorand-sdk/pull/301
* With the as is conventions, the encouraged way to make changes without breaking SDKs is to make 2-step changes as described below.
* In an algorand-sdk-testing PR, duplicate the test in-question. Assign a unique annotation.
* Merge the PR. SDKs are _safe_ because:
* New test is _not_ run until the annotation is configured.
* All other tests are unchanged.
* After making + merging changes to all SDKs, remove the test that was duplicated.

## Solution

Unclear - From group discussion, at least a couple of paths exist. **Bolded proposal** is preferred though more discussion is needed.
* Modify algorand-sdk-testing build process to test downstream SDKs. Implies failing the build when a downstream SDK's tests fail.
* There's a circular dependency between algorand-sdk-testing and SDKs that complicates the checks.
* It's possible to imagine a build process like:
* algorand-sdk-testing build looks for SDK branch names matching algorand-sdk-testing branch. If none found, use the SDK repo's default branch.
* Run integration tests on all SDKs.
* Concerns:
* Some algorand-sdk-testing changes have _no_ downstream consequences. The process must work for such scenarios.
* The proposed process feels heavyweight. It likely results in a _long_ build time.
* **Make SDKs depend on a tag of algorand-sdk-testing instead of `master`**. Implies implementing a process to ensure SDK repos update algorand-sdk-testing on release.
* Allows for looser coupling among SDK changes. _Should_ support retaining a simpler build process.
* It's possible to imagine starting with a manual process for keeping tags in-sync. In principle, stories _won't_ be closed until corresponding SDK changes complete.
* Constraint: All SDKs provide the same approach for configuring algorand-sdk-testing git tag/branch _without_ requiring hand-modified shell script modifications.

Stopgap measures until taking on an approach from above:
* Setup a nightly build for each SDK.
* If an algorand-sdk-testing PR breaks an SDK, we'll know within 24 hours rather than when someone else attempts to build.
* The process _feels_ straightforward to implement. And provides more visibility than we have today.
* Once a more principled approach is implemented, we can remove the nightly builds.

## Dependencies

N/A

## Urgency

TBD

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。