aws / aws/aws-durable-execution-sdk-python

[Testing] [ENH] Pre-publish CI: run aws-sam-cli durable integration tests against candidate emulator image

Đang mở
#430 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
pkg:testing
Ngôn ngữ chính
Python
Star
53
Fork
25
Merge trung bình
1 ngày 20 giờ
Pull request đã merge (30 ngày)
39

Mô tả

Migration from https://github.com/aws/aws-durable-execution-sdk-python-testing/issues/225

---

## Background

The emulator image `public.ecr.aws/durable-functions/aws-durable-execution-emulator:latest` is consumed automatically by [`aws-sam-cli`](https://github.com/aws/aws-sam-cli): on every `sam local invoke` of a durable function, sam-cli pulls `:latest` and refreshes the local cache (see [`durable_functions_emulator_container.py`](https://github.com/aws/aws-sam-cli/blob/develop/samcli/local/docker/durable_functions_emulator_container.py#L142-L148); customers can override per-invoke with `DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG` but the default is `:latest`). This means any image we publish ships immediately to every durable-functions customer running sam-cli, with no version pin in between.

PR #216 in this repo recently demonstrated the blast radius: ~26 sam-cli durable integration tests went red across `local-invoke`, `local-start-lambda`, `tier1-finch`, and `tier1-windows-other` jobs (e.g. [aws-sam-cli Integration Tests #496](https://github.com/aws/aws-sam-cli/actions/runs/26242969460/job/77233871806), [run #8779 / local-start-lambda](https://github.com/aws/aws-sam-cli/actions/runs/26249528554/job/77256683498)) the moment v1.2.0 went to `:latest`. Customer-visible symptom: a fresh `samdev local invoke` against any durable function 500s on first checkpoint or 404s on first `local execution get|history|stop|callback`. Mitigations are in flight on the sam-cli side (aws/aws-sam-cli#9038 merged, #9040 open) but they do not address the class problem: this repo's release pipeline has no signal from sam-cli before publishing `:latest`.

## Why our existing tests didn't catch this

`tests/web/e2e/routes_arn_encoding_int_test.py` (added in #222) drives a real boto client against this repo's `WebServer` and would have caught the emulator-side routing bug. It does not — and cannot — exercise sam-cli's `LocalLambdaHttpService`, which is a separate Flask service that customers' boto clients actually hit when using `samdev local invoke`. Anything we change in the ARN, callback ID, or function-qualifier shape can break sam-cli's service without touching ours.

## Proposal

Add a pre-publish CI step that builds the candidate emulator image and runs sam-cli's durable integration suite against it. Concrete shape:

1. Build the emulator image from this repo (we already do this in [`ecr-release.yml`](https://github.com/aws/aws-durable-execution-sdk-python-testing/blob/main/.github/workflows/ecr-release.yml)).
2. Tag it locally with a candidate tag, e.g. `aws-durable-execution-emulator:pr-${SHA}`.
3. Check out `aws/aws-sam-cli` at `develop`, install in `SAM_CLI_DEV=1` mode.
4. Run, with `DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG=pr-${SHA}`:
```bash
pytest -vv \
tests/integration/local/invoke/test_invoke_durable.py \
tests/integration/local/start_lambda/test_start_lambda_durable.py \
tests/integration/local/execution/test_execution.py \
tests/integration/local/callback/test_callback.py
```
That's the durable subset — ~50 tests, runs in ~3–5 min in CI based on the local-invoke and local-start-lambda timings above.
5. Publish to ECR only if step 4 is green.

Gate this on PRs that touch `src/**` so we get the signal pre-merge as well as pre-publish.

## Acceptance criteria

- A workflow (e.g. `.github/workflows/sam-cli-compat.yml`) that runs the four sam-cli durable test files against the locally-built emulator image and is required for PRs that change `src/`.
- The publish job (`ecr-release.yml`) gated on the same workflow's success.
- A README / CONTRIBUTING note explaining that any change affecting the emulator's HTTP contract — ARN shape, callback-token shape, route layout, response codes — must keep this job green.

## Out of scope

- Pinning sam-cli to a specific emulator tag. That just inverts the dependency: customers stop picking up emulator fixes until sam-cli ships a new release. Roll-forward + this CI gate is the durable answer.
- Running the full sam-cli integration suite. The four files above cover every code path that talks to the emulator.

## References

- Lambda public-API `DurableExecutionArn` shape: `arn::lambda:::function::/durable-execution//` ([API_GetDurableExecution](https://docs.aws.amazon.com/lambda/latest/api/API_GetDurableExecution.html))
- aws/aws-sam-cli#9037 — issue: `[Bug]: Durable integration tests can't extract execution ARN that contains "/"`
- aws/aws-sam-cli#9038 — PR: `fix(tests): accept '/' in durable execution ARN regex`
- aws/aws-sam-cli#9039 — issue: `[Bug]: Local Lambda HTTP service rejects DurableExecutionArn / CallbackId containing "/"`
- aws/aws-sam-cli#9040 — PR: `fix(local-lambda): accept documented Lambda DurableExecutionArn shape`
- This repo's #216 (introduced the `/`), #222 (URL-decode in own WebServer)

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Bắt đầu với .github/workflows/ecr-release.yml và bốn tệp kiểm thử durable được nêu tên của aws-sam-cli. Build và gắn tag cục bộ cho image emulator ứng viên, sau đó chạy subset pytest được chỉ định với SAM_CLI_DEV=1 và DURABLE_EXECUTIONS_EMULATOR_IMAGE_TAG được thiết lập. Hoàn thành nghĩa là workflow là bắt buộc đối với các thay đổi trong src/, ecr-release.yml được điều kiện hóa theo thành công của workflow đó, và README hoặc ghi chú CONTRIBUTING ghi lại yêu cầu về khả năng tương thích.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
docker, github-actions, python
Lĩnh vực
ci-cd, devops, release, testing
Loại issue
Tính năng
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Đặc tả rõ ràng
Mức phù hợp với người mới
55/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.