loopbackio / loopbackio/loopback-next

Move slow tests from `npm test` to a different test task

オープン
#5,670 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る

まだ誰も着手していません。

epic feature Internal Tooling
主要言語
TypeScript
スター
5.1k
フォーク
1.1k
平均マージ
2日 21時間
マージ済み PR(30日)
27

説明

Most of our test suite is running pretty fast, with individual tests finishing under 100ms. However, there are also several slow tests that take more than 1 second to finish. These tests are typically end-to-end/acceptance-level tests that are running running child processes, `npm install` and so on. As a result, `npm test` takes too long to complete (about 3 minutes on my machine).

Let's find a way how to reorganize our code base so that these slow tests can be put into "acceptance" or "end-to-end" group, excluded from `npm test` and have a dedicated CI job to run them in parallel with other CI tasks, as we are already doing for `acceptance/repository-mysql` and friends. (Alternatively, we can run the slow acceptance tests after the faster tests passed succesfully.)

1. For example, we can introduce new package(s) in the `acceptance` folder, e.g. `acceptance/cli-e2e-tests` and move the slow tests there. This should be easy to implement, but requires more work to introduce e2e tests for new packages.
2. Create a new directory-layout convention for acceptance/end-to-end tests, so that we can keep e2e tests inside each package alongside other tests, but still be able to exclude these tests from monorepo-level `npm test`. For example, we can introduce `src/__acceptance__` directory for such tests.

Personally, I think the second option is much better.

## List of very slow tests

Tests that take longer than one second to complete, sorted from the slowest:

- app-generator (SLOW) passes `npm test` for the generated project: 12441ms
- Benchmark (SLOW) works: 4352ms
- lb4 relation generates model relation for existing property name verifies that a preexisting property will be overwritten: 2675ms
- lb4 relation add new controller to existing index file check if the controller exported to index file : 2026ms
- lb4 relation add controller to existing index file only once check if the controller exported to index file only once: 1961ms
- cloneExampleFromGitHub (SLOW) extracts project files: 1869ms
- lb4 relation updates property decorator when property already exist in the model: 1566ms
- lb4 relation HasOne rejects relation when source key already exist in the model: 1362ms
- lb4 relation HasOne Execute relation with existing relation name rejects if the relation name already exists in the repository: 1344ms
- lb4 relation HasMany Execute relation with existing relation name rejects if the relation name already exists in the repository: 1325ms
- lb4 relation rejects relation when destination model doesn't have primary Key: 1320ms
- tsdocs runs api-extractor: 1280ms
- lb4 relation HasMany rejects relation when source key already exist in the model: 1254ms
- lb4 copyright with git updates copyright/license headers with options: 1239ms
- lb4 copyright for monorepo updates copyright/license headers: 1205ms
- build compiles ts files: 1046ms
- Application rejects division by zero with 412 error: 1022ms

## Nice to have

- Investigate why are `lb4 relation` tests so slow. Do they have expensive setup? Can we find a more efficient way how to test the relation generator?

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

調査の方向性

まず、モノレポレベルの `npm test` 設定と、既存の `acceptance/repository-mysql` CI セットアップを調査します。一覧にある遅いテストを確認し、`acceptance/cli-e2e-tests` と `src/__acceptance__` のレイアウトの選択肢を比較します。`npm test` で高速なテストが実行されなくなり、専用の acceptance タスクまたは CI ジョブで遅いテストが実行されれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
nodejs, typescript
領域
build-system, ci-cd, testing-qa
issue の種類
機能追加
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。