airbytehq / airbytehq/airbyte-python-cdk
Improve error message when a persisted cursor value cannot be parsed (ConcurrentPerPartitionCursor._set_global_state)
- 主要語言
- Python
- 星號
- 26
- 分支
- 53
- 平均合併
- 2 天 6 小時
- 30 天內合併 PR
- 10
描述
## Problem
When a persisted cursor state value cannot be parsed with the stream's configured `cursor_datetime_formats`, `ConcurrentPerPartitionCursor._set_global_state()` lets the raw `ValueError` escape during stream construction. The sync dies before any record is read, and the user-facing failure is:
```
No format in ['%s', '%s'] matching 1698686995.0
```
Observed in production on a manifest-only source (`source-slack`, `threads` stream) where the stored global cursor value carried a fractional epoch (`"1698686995.0"`) while the stream declares `cursor_datetime_formats: ["%s"]`.
## Why the current message is poor
- Leaks implementation details (`%s` format tokens, an internal Python list) with no user-meaningful vocabulary.
- Does not say which stream, which cursor field, or that the value came from saved sync state.
- Offers no remediation (e.g. "refresh/clear the stream state").
- Classified as `system_error` / `failure_origin: source`, so it looks like a connector crash rather than an unparseable-state condition.
- Duplicate entries in the format list (`['%s', '%s']`) because `CustomFormatConcurrentStreamStateConverter.__init__` appends `datetime_format` to `input_datetime_formats`, which makes the message look nonsensical.
## Suggested improvements
1. Wrap the failure in an `AirbyteTracedException` with a `config_error`-style (non-`system_error`) failure type, naming the stream, the cursor field, the offending value, and the accepted formats, plus a remediation hint pointing at refreshing the stream's state.
2. Consider mirroring the tolerance already added for record cursor values in [#758](https://github.com/airbytehq/airbyte-python-cdk/pull/758) (`observe()` logs a warning and skips instead of raising) so a single unparseable *state* value degrades to a warning rather than failing the sync.
3. De-duplicate the formats reported in the message.
## References
- Raise site: `airbyte_cdk/sources/streams/concurrent/state_converters/datetime_stream_state_converter.py` (`CustomFormatConcurrentStreamStateConverter.parse_timestamp`)
- Call path: `ConcurrentPerPartitionCursor._set_initial_state` → `_set_global_state` → `parse_value`
- Prior art for tolerant handling: https://github.com/airbytehq/airbyte-python-cdk/pull/758
- Originating oncall issue: https://github.com/airbytehq/oncall/issues/13342
---
[Devin session](https://app.devin.ai/sessions/e76104a6da8443f48a2ff8c5748df5b4)
貢獻指南
研究方向
Start in airbyte_cdk/sources/streams/concurrent/state_converters/datetime_stream_state_converter.py, especially CustomFormatConcurrentStreamStateConverter.parse_timestamp, then follow ConcurrentPerPartitionCursor._set_initial_state → _set_global_state → parse_value. Compare the handling with observe() and prior art in #758. Done means unparseable persisted state produces the agreed user-facing outcome, with relevant stream, cursor, value, formats, and remediation context without duplicate formats.
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- backend
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 活躍
- 描述清晰度
- 基本清楚
- 新手友好度
- 48/100