AbsaOSS / AbsaOSS/EventGate

Upgrade the Lambda runtime from Python 3.13 to 3.14

Ouverte
#222 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
infrastructure type:tech-debt
Langage dominant
Python
Étoiles
4
Forks
0
Merge moyen
1 j 5 h
PR mergées (30 j)
8

Description

### Description of Technical Debt

The local toolchain and the deployed runtime are on different Python versions:

| Where | Version | Source |
|---|---|---|
| Local venv / test runs | 3.14.7 | `./.venv/Scripts/python.exe --version` |
| Container image | 3.13 | `Dockerfile`: `public.ecr.aws/lambda/python:3.13-arm64` |
| mypy target | 3.13 | `pyproject.toml` `[tool.mypy] python_version` |
| black target | py313 | `pyproject.toml` `[tool.black] target-version` |

Every local `pytest`, `mypy`, `pylint`, and `black` run therefore executes under a different interpreter than production, and nothing in CI closes the gap.

Surfaced during review of PR #204, where the difference in annotation semantics between the two versions came up directly.

### Impact of Technical Debt

- **Local green does not prove production green.** Anything that changed between 3.13 and 3.14 passes locally and fails on deploy. Annotation evaluation is the concrete example: PEP 649 makes annotations lazy by default in 3.14 but not in 3.13, so a forward reference that resolves on a developer machine can raise `NameError` in Lambda.
- **The static analysis is checking the wrong target.** mypy is told `python_version = "3.13"` while running on a 3.14 interpreter, so its view of the stdlib and the developer's actual runtime disagree.
- **Review time is spent on it.** The 3.13-vs-3.14 distinction already consumed a PR #204 thread, and will keep resurfacing while the two differ.
- **The divergence widens on its own.** Nothing pins the local version, so a fresh venv on a newer interpreter moves further from production without anyone noticing.

### Category

DevOps / Infrastructure

### Priority

Medium - Should be addressed soon

### Proposed Solution

Move the deployed runtime to 3.14 so it matches what the project is already developed and tested on. The code already runs there — 290 unit tests pass locally on 3.14.7 — so this is a deployment-target change rather than a migration.

Verified against ECR Public and PyPI:

- `public.ecr.aws/lambda/python` publishes GA arm64 builds for 3.14 — 37 non-preview arm64 tags, latest `3.14.2026.08.26.13-arm64`.
- There is **no floating `3.14-arm64` tag**, unlike `3.13-arm64`. The plain `3.14` tag is a single-arch manifest, not a multi-arch list, so it cannot stand in for an arm64 build. The Dockerfile must pin a dated arm64 tag.
- Compiled dependencies publish cp314 artifacts: `confluent-kafka==2.15.0` (5 cp314 wheels), `psycopg2==2.9.12` (cp314 wheel, 3.14 classifier), `cryptography==50.0.0` (13 cp314 wheels, 3.14 classifier). `aws-lambda-powertools==3.31.1` declares the 3.14 classifier.

Work:

1. `Dockerfile`: `FROM --platform=linux/arm64 public.ecr.aws/lambda/python:3.14.-arm64`.
2. `pyproject.toml`: `[tool.mypy] python_version = "3.14"`, `[tool.black] target-version = ['py314']`.
3. Rebuild the image and confirm the source builds still succeed. This is the main risk: the Dockerfile passes `--no-binary confluent-kafka` to compile against the librdkafka 2.15.0 built in the image, so the published cp314 wheels are **not** used and the C extension is compiled against the 3.14 C API directly. `psycopg2` (not `psycopg2-binary`) compiles from source too.
4. Confirm the remaining pure-Python pins install cleanly on 3.14: `jsonschema`, `PyJWT`, `requests`, `boto3`/`botocore`, `aiosql`.
5. Run the integration tests against the rebuilt image, not just unit tests, so the Kafka and Postgres paths exercise the recompiled extensions.
6. Check whether CI pins a Python version anywhere and update it, so the mismatch cannot silently reappear.

**Alternative:** keep the runtime on 3.13 and recreate the venv on 3.13 instead, pinning the version in CI. That closes the gap equally well and avoids a runtime bump, at the cost of developing against an older interpreter. Either direction is acceptable; leaving the two apart is not.

### Effort Estimate

1–2 days, dominated by rebuilding the image and validating the compiled extensions

### Dependencies / Related

- PR #204, where this surfaced
- #193

### Additional Context

Done when:

- The image builds on 3.14 with librdkafka and psycopg2 compiled from source.
- Unit and integration tests pass against the rebuilt image.
- No tooling config still names 3.13.
- A deployed smoke test covers one `POST /topics/{topic_name}` fan-out and one `/health` probe.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.