AbsaOSS / AbsaOSS/EventGate

Upgrade the Lambda runtime from Python 3.13 to 3.14

Aberta
#222 0 comentários 0 reações 0 responsáveis Ver no GitHub
infrastructure type:tech-debt
Linguagem predominante
Python
Estrelas
4
Forks
0
Merge médio
1d 5h
PRs com merge (30d)
8

Descrição

### Description of Technical Debt

The local toolchain and the deployed runtime are on different Python versions:

| Where | Version | Source |
|---|---|---|
| Local venv / test runs | 3.14.7 | `./.venv/Scripts/python.exe --version` |
| Container image | 3.13 | `Dockerfile`: `public.ecr.aws/lambda/python:3.13-arm64` |
| mypy target | 3.13 | `pyproject.toml` `[tool.mypy] python_version` |
| black target | py313 | `pyproject.toml` `[tool.black] target-version` |

Every local `pytest`, `mypy`, `pylint`, and `black` run therefore executes under a different interpreter than production, and nothing in CI closes the gap.

Surfaced during review of PR #204, where the difference in annotation semantics between the two versions came up directly.

### Impact of Technical Debt

- **Local green does not prove production green.** Anything that changed between 3.13 and 3.14 passes locally and fails on deploy. Annotation evaluation is the concrete example: PEP 649 makes annotations lazy by default in 3.14 but not in 3.13, so a forward reference that resolves on a developer machine can raise `NameError` in Lambda.
- **The static analysis is checking the wrong target.** mypy is told `python_version = "3.13"` while running on a 3.14 interpreter, so its view of the stdlib and the developer's actual runtime disagree.
- **Review time is spent on it.** The 3.13-vs-3.14 distinction already consumed a PR #204 thread, and will keep resurfacing while the two differ.
- **The divergence widens on its own.** Nothing pins the local version, so a fresh venv on a newer interpreter moves further from production without anyone noticing.

### Category

DevOps / Infrastructure

### Priority

Medium - Should be addressed soon

### Proposed Solution

Move the deployed runtime to 3.14 so it matches what the project is already developed and tested on. The code already runs there — 290 unit tests pass locally on 3.14.7 — so this is a deployment-target change rather than a migration.

Verified against ECR Public and PyPI:

- `public.ecr.aws/lambda/python` publishes GA arm64 builds for 3.14 — 37 non-preview arm64 tags, latest `3.14.2026.08.26.13-arm64`.
- There is **no floating `3.14-arm64` tag**, unlike `3.13-arm64`. The plain `3.14` tag is a single-arch manifest, not a multi-arch list, so it cannot stand in for an arm64 build. The Dockerfile must pin a dated arm64 tag.
- Compiled dependencies publish cp314 artifacts: `confluent-kafka==2.15.0` (5 cp314 wheels), `psycopg2==2.9.12` (cp314 wheel, 3.14 classifier), `cryptography==50.0.0` (13 cp314 wheels, 3.14 classifier). `aws-lambda-powertools==3.31.1` declares the 3.14 classifier.

Work:

1. `Dockerfile`: `FROM --platform=linux/arm64 public.ecr.aws/lambda/python:3.14.-arm64`.
2. `pyproject.toml`: `[tool.mypy] python_version = "3.14"`, `[tool.black] target-version = ['py314']`.
3. Rebuild the image and confirm the source builds still succeed. This is the main risk: the Dockerfile passes `--no-binary confluent-kafka` to compile against the librdkafka 2.15.0 built in the image, so the published cp314 wheels are **not** used and the C extension is compiled against the 3.14 C API directly. `psycopg2` (not `psycopg2-binary`) compiles from source too.
4. Confirm the remaining pure-Python pins install cleanly on 3.14: `jsonschema`, `PyJWT`, `requests`, `boto3`/`botocore`, `aiosql`.
5. Run the integration tests against the rebuilt image, not just unit tests, so the Kafka and Postgres paths exercise the recompiled extensions.
6. Check whether CI pins a Python version anywhere and update it, so the mismatch cannot silently reappear.

**Alternative:** keep the runtime on 3.13 and recreate the venv on 3.13 instead, pinning the version in CI. That closes the gap equally well and avoids a runtime bump, at the cost of developing against an older interpreter. Either direction is acceptable; leaving the two apart is not.

### Effort Estimate

1–2 days, dominated by rebuilding the image and validating the compiled extensions

### Dependencies / Related

- PR #204, where this surfaced
- #193

### Additional Context

Done when:

- The image builds on 3.14 with librdkafka and psycopg2 compiled from source.
- Unit and integration tests pass against the rebuilt image.
- No tooling config still names 3.13.
- A deployed smoke test covers one `POST /topics/{topic_name}` fan-out and one `/health` probe.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.