loopbackio / loopbackio/loopback-next
Support `inq` splitting in `findByForeignKeys`
まだ誰も着手していません。
- 主要言語
- TypeScript
- スター
- 5.1k
- フォーク
- 1.1k
- 平均マージ
- 2日 21時間
- マージ済み PR(30日)
- 27
説明
Under the hood, inclusion resolvers are implemented using `inq` operator:
1. Gather PK/FK values from source models.
2. Query target models using `inq` and PK/FK values from step 1.
3. Assign target models to navigational property in source models.
This can be problematic when the number of source instances is large, we don't
know if all databases support `inq` with arbitrary number of items.
To address this issue, LB3 is implementing "inq splitting", where a single query
with arbitrary-sized `inq` condition is split into multiple queries where each
query has a reasonably-sized `inq` condition.
Connectors are allowed to specify the maximum `inq` size supported by the
database via `dataSource.settings.inqLimit` option. By default, `inqLimit` is
set to 256.
In this task, we need to improve `findByForeignKeys` (see https://github.com/strongloop/loopback-next/issues/3443) to handle the maximum size of `inq` parameter supported by the target database (data-source). When the list of provided FK values is too long, then we should split it into smaller chunks and execute multiple queries.
However, because our `Repository` interface is generic and does not assume that
a repository has to be backed by a data-source, I am proposing to expose
`inqLimit` via a new property of the `Repository` interface instead of accessing
the parameter via DataSource settings.
```ts
/**
* Description of capabilities offered by the connector backing the given
* datasource.
*/
export interface RepositoryCapabilities {
/**
* Maximum number of items allowed for `inq` operators.
* This value is used to split queries when resolving related models
* for a large number of source instances.
*/
inqLimit?: number;
}
```
To preserve backwards compatibility with existing repository implementation, we
cannot add `RepositoryCapabilities` directly to the `Repository` class. We need
to introduce a new interface instead that Repositories can (or may not)
implement.
```ts
export interface RepositoryWithCapabilities {
capabilities: RepositoryCapabilities;
}
```
See #3387 for more details & a prototype implementation.
## Acceptance criteria
To allow the helper to detect `inqLimit`, we need to extend Repository interfaces.
- [ ] Introduce `RepositoryCapabilities` interface (called `ConnectorCapabilities` in the spike), this interface will have a single property `inqLimit` (for now).
- [ ] Introduce `RepositoryWithCapabilities` interface (called `WithCapabilities` in the spike),= this interface should define `capabilities` property.
- [ ] Implement `isRepositoryWithCapabilities` type guard
- [ ] Implement `getRepositoryCapabilities` helper
The rest should be straightforward:
- [ ] Modify `findByForeignKeys` to obtain `inqLimit` from repository capabilities and implement query splitting (see the spike implementation).
- [ ] Write unit-level tests where we verify what (and how many) queries are called by `findByForeignKeys`.
- [ ] Write integration-level tests (in `repository-tests`) to verify that connectors can handle `inqLimit` they are advertising. For example, create a test that runs a query returning 1000 records.
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
調査の方向性
findByForeignKeys と repository interfaces から始め、次に #3387 で参照されているプロトタイプを確認します。リポジトリの機能がどのように検出されるかを追跡し、unit と integration のカバレッジを追加する前に repository-tests の領域を調べます。完了の条件は、機能用のヘルパーが存在し、大きな外部キーのリストが inqLimit に従って分割され、想定されるクエリ数が検証されることです。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- typescript
- 領域
- backend, testing
- issue の種類
- 機能追加
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100