aws / aws/aws-durable-execution-sdk-python

Plugin conformance tests are flaky when CloudWatch logs are not yet visible

Ouverte
#708 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
bug pkg:testing
Langage dominant
Python
Étoiles
53
Forks
25
Merge moyen
1 j 19 h
PR mergées (30 j)
40

Description

## Summary

The `conformance (plugin)` job can fail because expected plugin log records are not returned by CloudWatch, even though the durable execution and history validation complete successfully.

## Evidence

- Previous successful run: `33907972521` at commit `4ce54bcf`.
- Failed run: `33922277509` at commit `676c75a`.
- Attempt 1, job `101185043031`: test `10-11` (`plugin_nested_parent_linkage`) failed because both expected `operation-end` records matched 0 logs: the inner step record and the child-context record.
- Retry attempt 2, job `101188790974`: test `10-11` passed, but unrelated test `10-7` (`plugin_terminal_failure`) failed because its `invocation-start` and `invocation-end` records matched 0 logs.

The failure moving between tests on the same commit strongly suggests CloudWatch ingestion/filter timing or log-stream lookup flakiness rather than a deterministic SDK regression. The diff from the previous successful commit only changes OTel release metadata and does not affect the regular core plugin suite.

## Suggested investigation

- Increase or make configurable the CloudWatch log propagation/polling window in the conformance runner.
- Log the resolved log group, execution ARN filter, event count, and final retrieved records when `ExpectedLogs` fails.
- Pin the conformance runner to a known commit in CI instead of installing `@main`.
- Re-run the plugin suite repeatedly against the same commit to confirm the failure rate.

Relevant workflow: `.github/workflows/conformance-tests.yml`, step `Run conformance suite`.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez par .github/workflows/conformance-tests.yml et l’étape « Run conformance suite », puis relancez plusieurs fois la plugin suite sur le même commit. Étudiez la propagation de CloudWatch ou le délai de recherche du flux de logs et recueillez les diagnostics suggérés concernant le log-group, l’execution-filter, l’event-count et le retrieved-record; c’est terminé lorsque les conformance tests trouvent de manière fiable les enregistrements attendus.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
aws, python
Domaine
ci-cd, observability, testing
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Active
Clarté
Plutôt claire
Accessibilité débutants
48/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.