apache / apache/airflow

Record mapped_length for TypeScript SDK tasks with mapped dependants

Open
#71,063 0 comments 0 reactions 1 assignee Claimed by @jason810496 View on GitHub
area:core area:ts-sdk kind:feature on hold
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Background

A TypeScript task's return value cannot currently feed a downstream `.expand()`. The Python task runner handles this in `_push_xcom_if_needed`, but a foreign runtime cannot inspect the Dag to learn its output has mapped dependants.

https://github.com/apache/airflow/pull/70571 introduces the server-derived `TIRunContext.has_mapped_dependants` flag; when set, the supervisor records `mapped_length = len(value)` on the return-value `SetXCom` on the task's behalf.

### What needs to happen

1. Verify the supervisor-side recording from https://github.com/apache/airflow/pull/70571 applies to the `NodeCoordinator` path unchanged (it should, being supervisor-side rather than runtime-side).
2. Surface `hasMappedDependants` on the TypeScript `TaskContext` where useful to handler authors, following the existing camelCase mapping convention.
3. Cover the flow end to end: a TypeScript `@task.stub` returning an array, with a downstream Python `.expand()` over it.

### Acceptance criteria

- A TypeScript stub task whose return value feeds a downstream `.expand()` causes those dependants to expand to the array's length.
- A TypeScript task with no mapped dependants records no `mapped_length` (no behaviour change).
- The camelCase to snake_case mapping for the new field is covered by the existing schema-mapping test suite.

### Context

- Depends on https://github.com/apache/airflow/pull/70570 and https://github.com/apache/airflow/pull/70571
- Schema-mapping test suite: https://github.com/apache/airflow/pull/69297

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.