aws / aws/aws-durable-execution-sdk-python
Plugin conformance tests are flaky when CloudWatch logs are not yet visible
- Langage dominant
- Python
- Étoiles
- 53
- Forks
- 25
- Merge moyen
- 1 j 19 h
- PR mergées (30 j)
- 40
Description
## Summary
The `conformance (plugin)` job can fail because expected plugin log records are not returned by CloudWatch, even though the durable execution and history validation complete successfully.
## Evidence
- Previous successful run: `33907972521` at commit `4ce54bcf`.
- Failed run: `33922277509` at commit `676c75a`.
- Attempt 1, job `101185043031`: test `10-11` (`plugin_nested_parent_linkage`) failed because both expected `operation-end` records matched 0 logs: the inner step record and the child-context record.
- Retry attempt 2, job `101188790974`: test `10-11` passed, but unrelated test `10-7` (`plugin_terminal_failure`) failed because its `invocation-start` and `invocation-end` records matched 0 logs.
The failure moving between tests on the same commit strongly suggests CloudWatch ingestion/filter timing or log-stream lookup flakiness rather than a deterministic SDK regression. The diff from the previous successful commit only changes OTel release metadata and does not affect the regular core plugin suite.
## Suggested investigation
- Increase or make configurable the CloudWatch log propagation/polling window in the conformance runner.
- Log the resolved log group, execution ARN filter, event count, and final retrieved records when `ExpectedLogs` fails.
- Pin the conformance runner to a known commit in CI instead of installing `@main`.
- Re-run the plugin suite repeatedly against the same commit to confirm the failure rate.
Relevant workflow: `.github/workflows/conformance-tests.yml`, step `Run conformance suite`.
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Commencez par .github/workflows/conformance-tests.yml et l’étape « Run conformance suite », puis relancez plusieurs fois la plugin suite sur le même commit. Étudiez la propagation de CloudWatch ou le délai de recherche du flux de logs et recueillez les diagnostics suggérés concernant le log-group, l’execution-filter, l’event-count et le retrieved-record; c’est terminé lorsque les conformance tests trouvent de manière fiable les enregistrements attendus.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- aws, python
- Domaine
- ci-cd, observability, testing
- Type d'issue
- Bug
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- Active
- Clarté
- Plutôt claire
- Accessibilité débutants
- 48/100