aws / aws/aws-durable-execution-sdk-python
[Testing] [ENH] Pre-publish CI: run aws-sam-cli durable integration tests against candidate emulator image
- 主要言語
- Python
- スター
- 53
- フォーク
- 25
- 平均マージ
- 1日 19時間
- マージ済み PR(30日)
- 40
説明
Migration from https://github.com/aws/aws-durable-execution-sdk-python-testing/issues/225
---
## Background
The emulator image `public.ecr.aws/durable-functions/aws-durable-execution-emulator:latest` is consumed automatically by [`aws-sam-cli`](https://github.com/aws/aws-sam-cli): on every `sam local invoke` of a durable function, sam-cli pulls `:latest` and refreshes the local cache (see [`durable_functions_emulator_container.py`](https://github.com/aws/aws-sam-cli/blob/develop/samcli/local/docker/durable_functions_emulator_container.py#L142-L148); customers can override per-invoke with `DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG` but the default is `:latest`). This means any image we publish ships immediately to every durable-functions customer running sam-cli, with no version pin in between.
PR #216 in this repo recently demonstrated the blast radius: ~26 sam-cli durable integration tests went red across `local-invoke`, `local-start-lambda`, `tier1-finch`, and `tier1-windows-other` jobs (e.g. [aws-sam-cli Integration Tests #496](https://github.com/aws/aws-sam-cli/actions/runs/26242969460/job/77233871806), [run #8779 / local-start-lambda](https://github.com/aws/aws-sam-cli/actions/runs/26249528554/job/77256683498)) the moment v1.2.0 went to `:latest`. Customer-visible symptom: a fresh `samdev local invoke` against any durable function 500s on first checkpoint or 404s on first `local execution get|history|stop|callback`. Mitigations are in flight on the sam-cli side (aws/aws-sam-cli#9038 merged, #9040 open) but they do not address the class problem: this repo's release pipeline has no signal from sam-cli before publishing `:latest`.
## Why our existing tests didn't catch this
`tests/web/e2e/routes_arn_encoding_int_test.py` (added in #222) drives a real boto client against this repo's `WebServer` and would have caught the emulator-side routing bug. It does not — and cannot — exercise sam-cli's `LocalLambdaHttpService`, which is a separate Flask service that customers' boto clients actually hit when using `samdev local invoke`. Anything we change in the ARN, callback ID, or function-qualifier shape can break sam-cli's service without touching ours.
## Proposal
Add a pre-publish CI step that builds the candidate emulator image and runs sam-cli's durable integration suite against it. Concrete shape:
1. Build the emulator image from this repo (we already do this in [`ecr-release.yml`](https://github.com/aws/aws-durable-execution-sdk-python-testing/blob/main/.github/workflows/ecr-release.yml)).
2. Tag it locally with a candidate tag, e.g. `aws-durable-execution-emulator:pr-${SHA}`.
3. Check out `aws/aws-sam-cli` at `develop`, install in `SAM_CLI_DEV=1` mode.
4. Run, with `DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG=pr-${SHA}`:
```bash
pytest -vv \
tests/integration/local/invoke/test_invoke_durable.py \
tests/integration/local/start_lambda/test_start_lambda_durable.py \
tests/integration/local/execution/test_execution.py \
tests/integration/local/callback/test_callback.py
```
That's the durable subset — ~50 tests, runs in ~3–5 min in CI based on the local-invoke and local-start-lambda timings above.
5. Publish to ECR only if step 4 is green.
Gate this on PRs that touch `src/**` so we get the signal pre-merge as well as pre-publish.
## Acceptance criteria
- A workflow (e.g. `.github/workflows/sam-cli-compat.yml`) that runs the four sam-cli durable test files against the locally-built emulator image and is required for PRs that change `src/`.
- The publish job (`ecr-release.yml`) gated on the same workflow's success.
- A README / CONTRIBUTING note explaining that any change affecting the emulator's HTTP contract — ARN shape, callback-token shape, route layout, response codes — must keep this job green.
## Out of scope
- Pinning sam-cli to a specific emulator tag. That just inverts the dependency: customers stop picking up emulator fixes until sam-cli ships a new release. Roll-forward + this CI gate is the durable answer.
- Running the full sam-cli integration suite. The four files above cover every code path that talks to the emulator.
## References
- Lambda public-API `DurableExecutionArn` shape: `arn::lambda:::function::/durable-execution//` ([API_GetDurableExecution](https://docs.aws.amazon.com/lambda/latest/api/API_GetDurableExecution.html))
- aws/aws-sam-cli#9037 — issue: `[Bug]: Durable integration tests can't extract execution ARN that contains "/"`
- aws/aws-sam-cli#9038 — PR: `fix(tests): accept '/' in durable execution ARN regex`
- aws/aws-sam-cli#9039 — issue: `[Bug]: Local Lambda HTTP service rejects DurableExecutionArn / CallbackId containing "/"`
- aws/aws-sam-cli#9040 — PR: `fix(local-lambda): accept documented Lambda DurableExecutionArn shape`
- This repo's #216 (introduced the `/`), #222 (URL-decode in own WebServer)
コントリビューションガイド
調査の方向性
.github/workflows/ecr-release.yml と、指定された aws-sam-cli の4つの durable テストファイルから始めます。候補エミュレーターイメージをビルドしてローカルでタグ付けし、SAM_CLI_DEV=1 と DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG を設定して、指定された pytest サブセットを実行します。完了条件は、src/ の変更でこの workflow が必須になっていること、ecr-release.yml がその成功を条件としていること、そして README または CONTRIBUTING の注記に互換性要件が記載されていることです。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- docker, github-actions, python
- 領域
- ci-cd, devops, release, testing
- issue の種類
- 機能追加
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 静か
- 明瞭さ
- 明確に書かれている
- 初心者へのやさしさ
- 55/100