airbytehq / airbytehq/PyAirbyte

Sync mode selection should validate against stream's supported_sync_modes

Đang mở
#917 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
344
Fork
77
Merge trung bình
1 ngày 11 giờ
Pull request đã merge (30 ngày)
35

Mô tả

## Summary

When building the configured catalog, the `_get_sync_mode` helper function unconditionally falls back to `SyncMode.incremental`, even for streams that only support `full_refresh`. This can cause connector failures for streams that don't support incremental sync.

## Background

This issue was identified during code review of [PR #899](https://github.com/airbytehq/PyAirbyte/pull/899) (force_full_refresh fix). The current behavior predates that PR, so it's not a regression - but it's a valid edge case that should be addressed.

**Requested by:** @aaronsteers (AJ Steers)

## Current Behavior

```python
def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
supported_modes = getattr(stream, "supported_sync_modes", None)

if force_full_refresh:
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
return SyncMode.incremental # BUG: stream might not support incremental

return SyncMode.incremental # BUG: stream might only support full_refresh
```

## Expected Behavior

The selected sync mode should always be validated against `supported_sync_modes` when that attribute is present:

```python
def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
supported_modes = getattr(stream, "supported_sync_modes", None)

if force_full_refresh:
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
if supported_modes and SyncMode.incremental in supported_modes:
return SyncMode.incremental
return SyncMode.full_refresh # fallback for unknown/empty

# Default: prefer incremental if supported, otherwise full_refresh
if supported_modes and SyncMode.incremental in supported_modes:
return SyncMode.incremental
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
return SyncMode.incremental # preserve backwards compat for unknown
```

## Affected Code

- `airbyte/sources/base.py` - `_get_sync_mode` helper function (around line 449)

## References

- CodeRabbit comment: https://github.com/airbytehq/PyAirbyte/pull/899#discussion_r2626012395
- Related PR: https://github.com/airbytehq/PyAirbyte/pull/899

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.