airbytehq / airbytehq/PyAirbyte

Sync mode selection should validate against stream's supported_sync_modes

Abierto
#917 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
344
Forks
77
Merge medio
1 d 11 h
PR fusionados (30 d)
35

Descripción

## Summary

When building the configured catalog, the `_get_sync_mode` helper function unconditionally falls back to `SyncMode.incremental`, even for streams that only support `full_refresh`. This can cause connector failures for streams that don't support incremental sync.

## Background

This issue was identified during code review of [PR #899](https://github.com/airbytehq/PyAirbyte/pull/899) (force_full_refresh fix). The current behavior predates that PR, so it's not a regression - but it's a valid edge case that should be addressed.

**Requested by:** @aaronsteers (AJ Steers)

## Current Behavior

```python
def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
supported_modes = getattr(stream, "supported_sync_modes", None)

if force_full_refresh:
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
return SyncMode.incremental # BUG: stream might not support incremental

return SyncMode.incremental # BUG: stream might only support full_refresh
```

## Expected Behavior

The selected sync mode should always be validated against `supported_sync_modes` when that attribute is present:

```python
def _get_sync_mode(stream: AirbyteStream) -> SyncMode:
supported_modes = getattr(stream, "supported_sync_modes", None)

if force_full_refresh:
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
if supported_modes and SyncMode.incremental in supported_modes:
return SyncMode.incremental
return SyncMode.full_refresh # fallback for unknown/empty

# Default: prefer incremental if supported, otherwise full_refresh
if supported_modes and SyncMode.incremental in supported_modes:
return SyncMode.incremental
if supported_modes and SyncMode.full_refresh in supported_modes:
return SyncMode.full_refresh
return SyncMode.incremental # preserve backwards compat for unknown
```

## Affected Code

- `airbyte/sources/base.py` - `_get_sync_mode` helper function (around line 449)

## References

- CodeRabbit comment: https://github.com/airbytehq/PyAirbyte/pull/899#discussion_r2626012395
- Related PR: https://github.com/airbytehq/PyAirbyte/pull/899

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.