airbytehq / airbytehq/airbyte-python-cdk

Improve error message when a persisted cursor value cannot be parsed (ConcurrentPerPartitionCursor._set_global_state)

オープン
#1,120 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る
enhancement
主要言語
Python
スター
26
フォーク
53
平均マージ
2日 6時間
マージ済み PR(30日)
10

説明

## Problem

When a persisted cursor state value cannot be parsed with the stream's configured `cursor_datetime_formats`, `ConcurrentPerPartitionCursor._set_global_state()` lets the raw `ValueError` escape during stream construction. The sync dies before any record is read, and the user-facing failure is:

```
No format in ['%s', '%s'] matching 1698686995.0
```

Observed in production on a manifest-only source (`source-slack`, `threads` stream) where the stored global cursor value carried a fractional epoch (`"1698686995.0"`) while the stream declares `cursor_datetime_formats: ["%s"]`.

## Why the current message is poor

- Leaks implementation details (`%s` format tokens, an internal Python list) with no user-meaningful vocabulary.
- Does not say which stream, which cursor field, or that the value came from saved sync state.
- Offers no remediation (e.g. "refresh/clear the stream state").
- Classified as `system_error` / `failure_origin: source`, so it looks like a connector crash rather than an unparseable-state condition.
- Duplicate entries in the format list (`['%s', '%s']`) because `CustomFormatConcurrentStreamStateConverter.__init__` appends `datetime_format` to `input_datetime_formats`, which makes the message look nonsensical.

## Suggested improvements

1. Wrap the failure in an `AirbyteTracedException` with a `config_error`-style (non-`system_error`) failure type, naming the stream, the cursor field, the offending value, and the accepted formats, plus a remediation hint pointing at refreshing the stream's state.
2. Consider mirroring the tolerance already added for record cursor values in [#758](https://github.com/airbytehq/airbyte-python-cdk/pull/758) (`observe()` logs a warning and skips instead of raising) so a single unparseable *state* value degrades to a warning rather than failing the sync.
3. De-duplicate the formats reported in the message.

## References

- Raise site: `airbyte_cdk/sources/streams/concurrent/state_converters/datetime_stream_state_converter.py` (`CustomFormatConcurrentStreamStateConverter.parse_timestamp`)
- Call path: `ConcurrentPerPartitionCursor._set_initial_state` → `_set_global_state` → `parse_value`
- Prior art for tolerant handling: https://github.com/airbytehq/airbyte-python-cdk/pull/758
- Originating oncall issue: https://github.com/airbytehq/oncall/issues/13342

---
[Devin session](https://app.devin.ai/sessions/e76104a6da8443f48a2ff8c5748df5b4)

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Start in airbyte_cdk/sources/streams/concurrent/state_converters/datetime_stream_state_converter.py, especially CustomFormatConcurrentStreamStateConverter.parse_timestamp, then follow ConcurrentPerPartitionCursor._set_initial_state → _set_global_state → parse_value. Compare the handling with observe() and prior art in #758. Done means unparseable persisted state produces the agreed user-facing outcome, with relevant stream, cursor, value, formats, and remediation context without duplicate formats.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
backend
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
活発
明瞭さ
おおむね明確
初心者へのやさしさ
48/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。