Single Sign On E2E test concept

This document is subject of change at all time based on feedback in the comments.

ℹ️ Terms

• SSO lib: Nextcloud Single Sign On library
• 3rd party app: Any third party Android app the uses the SSO lib
• files app: The Nextcloud Android app
• server: A running Nextcloud instance that is accessible via Browser, REST, etc. ...

🙋‍♂ Motivation

The files app 3.18-RC1 introduced an issue which caused all 3rd party apps stop working. The aftermaths were complaining users, increased support efforts and 1-star ratings. We should avoid this in the future.

🏁 Goal

1. When changes in the files app or in the SSO lib lead to a breaking change, we must get notified
2. Having an real end to end test, which covers the flow of a 3rd party app authenticating and using the latest SSO lib in combination with the latest beta files app
3. Having an real end to end test, which covers the flow of a 3rd party app authenticating and using the latest stable SSO sample app in combination with the latest files app

➡️ Proposal

In order to achieve goal 2 and 3, i came to the conclusion that we eventually need two separate test flows. One should be located in the repository of the SSO lib and one in the repository of the files app.

Testing flows

We already know that those tests will have a quite long runtime due to the usage of an AVD emulator. Ideally they will run on each push event, but i am also fine with setting up a nightly running job. Maybe listening to push events makes sense for the SSO lib (where less commits are made, and we usually have more time), while setting up a nightly job for the files app.

Files app

1. Setup a temporary server using the Docker image
2. Run emulator
3. Fetch (or clone) and install the latest stable SSO sample app, for example via GitHub, jitpack.io or F-Droid (given the sample app will be published there)
4. Build and install the current files app
5. Run actual test

SSO lib

1. Setup a temporary server using the Docker image
2. Run emulator
3. Fetch and install the latest beta files app
4. Build and install the current files app
5. Run actual test

I would love to get some feedback from @tobiasKaminsky @David-Development @desperateCoder - also feel free to ping everyone who might be interested, wants to join forces or can provide further information (for example regarding Docker.

The issue names no repository files or existing tests; start by reviewing the proposed Docker server setup, emulator workflow example, and the two SSO testing flows. Done means defining and implementing end-to-end coverage for both the stable files-app flow and the beta files-app flow, with an agreed push or nightly workflow.