loopbackio / loopbackio/loopback-next
Move slow tests from `npm test` to a different test task
まだ誰も着手していません。
- 主要言語
- TypeScript
- スター
- 5.1k
- フォーク
- 1.1k
- 平均マージ
- 2日 21時間
- マージ済み PR(30日)
- 27
説明
Most of our test suite is running pretty fast, with individual tests finishing under 100ms. However, there are also several slow tests that take more than 1 second to finish. These tests are typically end-to-end/acceptance-level tests that are running running child processes, `npm install` and so on. As a result, `npm test` takes too long to complete (about 3 minutes on my machine).
Let's find a way how to reorganize our code base so that these slow tests can be put into "acceptance" or "end-to-end" group, excluded from `npm test` and have a dedicated CI job to run them in parallel with other CI tasks, as we are already doing for `acceptance/repository-mysql` and friends. (Alternatively, we can run the slow acceptance tests after the faster tests passed succesfully.)
1. For example, we can introduce new package(s) in the `acceptance` folder, e.g. `acceptance/cli-e2e-tests` and move the slow tests there. This should be easy to implement, but requires more work to introduce e2e tests for new packages.
2. Create a new directory-layout convention for acceptance/end-to-end tests, so that we can keep e2e tests inside each package alongside other tests, but still be able to exclude these tests from monorepo-level `npm test`. For example, we can introduce `src/__acceptance__` directory for such tests.
Personally, I think the second option is much better.
## List of very slow tests
Tests that take longer than one second to complete, sorted from the slowest:
- app-generator (SLOW) passes `npm test` for the generated project: 12441ms
- Benchmark (SLOW) works: 4352ms
- lb4 relation generates model relation for existing property name verifies that a preexisting property will be overwritten: 2675ms
- lb4 relation add new controller to existing index file check if the controller exported to index file : 2026ms
- lb4 relation add controller to existing index file only once check if the controller exported to index file only once: 1961ms
- cloneExampleFromGitHub (SLOW) extracts project files: 1869ms
- lb4 relation updates property decorator when property already exist in the model: 1566ms
- lb4 relation HasOne rejects relation when source key already exist in the model: 1362ms
- lb4 relation HasOne Execute relation with existing relation name rejects if the relation name already exists in the repository: 1344ms
- lb4 relation HasMany Execute relation with existing relation name rejects if the relation name already exists in the repository: 1325ms
- lb4 relation rejects relation when destination model doesn't have primary Key: 1320ms
- tsdocs runs api-extractor: 1280ms
- lb4 relation HasMany rejects relation when source key already exist in the model: 1254ms
- lb4 copyright with git updates copyright/license headers with options: 1239ms
- lb4 copyright for monorepo updates copyright/license headers: 1205ms
- build compiles ts files: 1046ms
- Application rejects division by zero with 412 error: 1022ms
## Nice to have
- Investigate why are `lb4 relation` tests so slow. Do they have expensive setup? Can we find a more efficient way how to test the relation generator?
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
調査の方向性
まず、モノレポレベルの `npm test` 設定と、既存の `acceptance/repository-mysql` CI セットアップを調査します。一覧にある遅いテストを確認し、`acceptance/cli-e2e-tests` と `src/__acceptance__` のレイアウトの選択肢を比較します。`npm test` で高速なテストが実行されなくなり、専用の acceptance タスクまたは CI ジョブで遅いテストが実行されれば完了です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- nodejs, typescript
- 領域
- build-system, ci-cd, testing-qa
- issue の種類
- 機能追加
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100