aws / aws/aws-durable-execution-sdk-python
[Testing] [ENH] Pre-publish CI: run aws-sam-cli durable integration tests against candidate emulator image
- Ngôn ngữ chính
- Python
- Star
- 53
- Fork
- 25
- Merge trung bình
- 1 ngày 20 giờ
- Pull request đã merge (30 ngày)
- 39
Mô tả
Migration from https://github.com/aws/aws-durable-execution-sdk-python-testing/issues/225
---
## Background
The emulator image `public.ecr.aws/durable-functions/aws-durable-execution-emulator:latest` is consumed automatically by [`aws-sam-cli`](https://github.com/aws/aws-sam-cli): on every `sam local invoke` of a durable function, sam-cli pulls `:latest` and refreshes the local cache (see [`durable_functions_emulator_container.py`](https://github.com/aws/aws-sam-cli/blob/develop/samcli/local/docker/durable_functions_emulator_container.py#L142-L148); customers can override per-invoke with `DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG` but the default is `:latest`). This means any image we publish ships immediately to every durable-functions customer running sam-cli, with no version pin in between.
PR #216 in this repo recently demonstrated the blast radius: ~26 sam-cli durable integration tests went red across `local-invoke`, `local-start-lambda`, `tier1-finch`, and `tier1-windows-other` jobs (e.g. [aws-sam-cli Integration Tests #496](https://github.com/aws/aws-sam-cli/actions/runs/26242969460/job/77233871806), [run #8779 / local-start-lambda](https://github.com/aws/aws-sam-cli/actions/runs/26249528554/job/77256683498)) the moment v1.2.0 went to `:latest`. Customer-visible symptom: a fresh `samdev local invoke` against any durable function 500s on first checkpoint or 404s on first `local execution get|history|stop|callback`. Mitigations are in flight on the sam-cli side (aws/aws-sam-cli#9038 merged, #9040 open) but they do not address the class problem: this repo's release pipeline has no signal from sam-cli before publishing `:latest`.
## Why our existing tests didn't catch this
`tests/web/e2e/routes_arn_encoding_int_test.py` (added in #222) drives a real boto client against this repo's `WebServer` and would have caught the emulator-side routing bug. It does not — and cannot — exercise sam-cli's `LocalLambdaHttpService`, which is a separate Flask service that customers' boto clients actually hit when using `samdev local invoke`. Anything we change in the ARN, callback ID, or function-qualifier shape can break sam-cli's service without touching ours.
## Proposal
Add a pre-publish CI step that builds the candidate emulator image and runs sam-cli's durable integration suite against it. Concrete shape:
1. Build the emulator image from this repo (we already do this in [`ecr-release.yml`](https://github.com/aws/aws-durable-execution-sdk-python-testing/blob/main/.github/workflows/ecr-release.yml)).
2. Tag it locally with a candidate tag, e.g. `aws-durable-execution-emulator:pr-${SHA}`.
3. Check out `aws/aws-sam-cli` at `develop`, install in `SAM_CLI_DEV=1` mode.
4. Run, with `DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG=pr-${SHA}`:
```bash
pytest -vv \
tests/integration/local/invoke/test_invoke_durable.py \
tests/integration/local/start_lambda/test_start_lambda_durable.py \
tests/integration/local/execution/test_execution.py \
tests/integration/local/callback/test_callback.py
```
That's the durable subset — ~50 tests, runs in ~3–5 min in CI based on the local-invoke and local-start-lambda timings above.
5. Publish to ECR only if step 4 is green.
Gate this on PRs that touch `src/**` so we get the signal pre-merge as well as pre-publish.
## Acceptance criteria
- A workflow (e.g. `.github/workflows/sam-cli-compat.yml`) that runs the four sam-cli durable test files against the locally-built emulator image and is required for PRs that change `src/`.
- The publish job (`ecr-release.yml`) gated on the same workflow's success.
- A README / CONTRIBUTING note explaining that any change affecting the emulator's HTTP contract — ARN shape, callback-token shape, route layout, response codes — must keep this job green.
## Out of scope
- Pinning sam-cli to a specific emulator tag. That just inverts the dependency: customers stop picking up emulator fixes until sam-cli ships a new release. Roll-forward + this CI gate is the durable answer.
- Running the full sam-cli integration suite. The four files above cover every code path that talks to the emulator.
## References
- Lambda public-API `DurableExecutionArn` shape: `arn::lambda:::function::/durable-execution//` ([API_GetDurableExecution](https://docs.aws.amazon.com/lambda/latest/api/API_GetDurableExecution.html))
- aws/aws-sam-cli#9037 — issue: `[Bug]: Durable integration tests can't extract execution ARN that contains "/"`
- aws/aws-sam-cli#9038 — PR: `fix(tests): accept '/' in durable execution ARN regex`
- aws/aws-sam-cli#9039 — issue: `[Bug]: Local Lambda HTTP service rejects DurableExecutionArn / CallbackId containing "/"`
- aws/aws-sam-cli#9040 — PR: `fix(local-lambda): accept documented Lambda DurableExecutionArn shape`
- This repo's #216 (introduced the `/`), #222 (URL-decode in own WebServer)
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu với .github/workflows/ecr-release.yml và bốn tệp kiểm thử durable được nêu tên của aws-sam-cli. Build và gắn tag cục bộ cho image emulator ứng viên, sau đó chạy subset pytest được chỉ định với SAM_CLI_DEV=1 và DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG được thiết lập. Hoàn thành nghĩa là workflow là bắt buộc đối với các thay đổi trong src/, ecr-release.yml được điều kiện hóa theo thành công của workflow đó, và README hoặc ghi chú CONTRIBUTING ghi lại yêu cầu về khả năng tương thích.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- docker, github-actions, python
- Lĩnh vực
- ci-cd, devops, release, testing
- Loại issue
- Tính năng
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 55/100