airbytehq / airbytehq/airbyte

Support the MAX_*_WORKERS environment variables in the Helm chart

オープン
#29,547 コメント 10 件 リアクション 6 件 担当者 0 名 GitHub で見る
area/platform autoteam community team/use type/enhancement
主要言語
Python
スター
22.1k
フォーク
5.3k
PR マージ指標
PR 指標を取得中

説明

### What area the feature impact?

Helm Chart

### Revelant Information

As we continue to scale Airbyte, we're running up against the five-worker default limits. The [documentation](https://docs.airbyte.com/operator-guides/configuring-airbyte#worker) suggests that we can set the following environment variables, all of which are defaulted to five:

* `MAX_CHECK_WORKERS`
* `MAX_DISCOVER_WORKERS`
* `MAX_SYNC_WORKERS`
* `MAX_SPEC_WORKERS`

However, in [the Helm chart](https://github.com/airbytehq/airbyte-platform/blob/56831ccae4657e5e43a81a1595a538b739d27ba9/charts/airbyte/templates/env-configmap.yaml#L96), only `MAX_NOTIFY_WORKERS` can be set, which is not shown in the documentation.

I attempted setting `MAX_NOTIFY_WORKERS`, but it does not seem do to anything for the number of concurrent workers we can run.

We'd like to be able to run more workers concurrently by setting these environment variables through Helm.

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。