Github CI is not useful for external contributors
- Dominant language
- Kotlin
- Stars
- 5.6k
- Forks
- 2k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 92
Description
As I am not an expert with Github CI (I am more familiar with Gitlab), parts of these might be Github's fault, but nonetheless - the CI is not doing any favor to external contributors. IMHO, it should support them better, to motivate more outside contributions
1. Email flood
- After a lot of quiet on the PR, I am suddenly bombarded with ~13 emails about failing jobs. Especially if I have multiple PRs open, it is quite hard to keep track of what is affected
- Any chance of having one email per PR instead, reporting the CI failure once, per PR?
-
2. It is unclear when the CI runs and where
- We should explain in the CONTRIBUTING.md that most jobs (but not all, I think?) run only when a maintainer starts them
- There is also a Drone CI running somewhere?
- It is unclear where I see my newly added tests actually running
- I tried to get to the bottom of this here, but didn't get far: https://github.com/nextcloud/android/pull/15424#discussion_r2296719115
3. Some jobs simply fail by design, when it is on a fork
- Example: https://github.com/nextcloud/android/actions/runs/20276837329/job/59030849217?pr=16145
- this leads to an unnecessary and wrongful impression that there is something wrong. Rather, any CI failure should directly map to an action item - but in these cases, there is nothing you can do.
4. It is unclear what checks should be run locally, to ensure a smooth CI experience
- E.g. `gradlew lint` is failing the CI, but it is not explained anywhere that you should run it. The pre-commit hook also doesn't execute it (probably because it is too expensive to run every time)
- Example: https://github.com/nextcloud/android/actions/runs/20276837368/job/59030849289?pr=16145
5. Details of a CI failure are hard to understand
- For example no full lint report downloadable on https://github.com/nextcloud/android/actions/runs/20276837368/job/59030849289?pr=16145
- For example no access to details for screenshot test failures: https://github.com/nextcloud/android/actions/runs/20276837409/job/59030849251?pr=16145
6. Said linter job ~just always~ fails, unrelated to the changes in the PR
- Example: https://github.com/nextcloud/android/actions/runs/20276837368/job/59030849289?pr=16145 fails due to issues in files the PR didn't even touch. So it is unclear, if and what I am supposed to do.
7. The `detectWrongSettings` job is destined to fail
- Because the maintainer triggering of the CI job is happening weeks later than the original PR, at time of triggering the PR is almost always out-of-date, so it doesn't use one of the latest 10 android-library commits.
- This issue is fixable by the contributor, by rebasing, but it does cause additional friction as well (yet another failure, yet another alert, ...)
cc @AndyScherzinger
Contributor guide
Assessment
This issue has not been assessed yet.