airbytehq / airbytehq/PyAirbyte

Sync mode selection should validate against stream's supported_sync_modes

オープン
#917 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
344
フォーク
77
平均マージ
1日 11時間
マージ済み PR(30日)
35

説明

## Summary

When building the configured catalog, the `_get_sync_mode` helper function unconditionally falls back to `SyncMode.incremental`, even for streams that only support `full_refresh`. This can cause connector failures for streams that don't support incremental sync.

## Background

This issue was identified during code review of [PR #899](https://github.com/airbytehq/PyAirbyte/pull/899) (force_full_refresh fix). The current behavior predates that PR, so it's not a regression - but it's a valid edge case that should be addressed.

**Requested by:** @aaronsteers (AJ Steers)

## Current Behavior

```python
def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
supported_modes = getattr(stream, "supported_sync_modes", None)

if force_full_refresh:
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
return SyncMode.incremental # BUG: stream might not support incremental

return SyncMode.incremental # BUG: stream might only support full_refresh
```

## Expected Behavior

The selected sync mode should always be validated against `supported_sync_modes` when that attribute is present:

```python
def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
supported_modes = getattr(stream, "supported_sync_modes", None)

if force_full_refresh:
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
if supported_modes and SyncMode.incremental in supported_modes:
return SyncMode.incremental
return SyncMode.full_refresh # fallback for unknown/empty

# Default: prefer incremental if supported, otherwise full_refresh
if supported_modes and SyncMode.incremental in supported_modes:
return SyncMode.incremental
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
return SyncMode.incremental # preserve backwards compat for unknown
```

## Affected Code

- `airbyte/sources/base.py` - `_get_sync_mode` helper function (around line 449)

## References

- CodeRabbit comment: https://github.com/airbytehq/PyAirbyte/pull/899#discussion_r2626012395
- Related PR: https://github.com/airbytehq/PyAirbyte/pull/899

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。