Java-SDK OpenLineage support (lineage callback from Java task to Python supervisor)

### Background

In the AIP-108 dev@ thread ([2026-05-13 reply][maciej-reply]), Maciej
Obuchowski outlined what OpenLineage needs from the Java SDK:

- **Generic task lifecycle events** — when OL is enabled, every task
execution emits OL start / complete / failure events via the listener
framework, driven by the Python task runner. For Java tasks, the listener
calls still fire from the Python side around the Java subprocess (e.g.
near `on_task_instance_running` / `on_task_instance_success` in
`task-sdk/src/airflow/sdk/execution_time/task_runner.py:1196` and
`:1921`). No code change is required to keep this working for the
"task ran, task succeeded" signal.
- **Operator/hook-specific lineage data** — the part that currently relies
on Python operators/hooks being in-process and reading state from the
`TaskInstance` after `execute()`. This does **not** work for a Java task
because the user code runs in a JVM subprocess that the Python listener
cannot introspect.

Maciej's conclusion: v1 of the Java SDK does not need OL emission from
inside the Java task, but the IPC and Java-side API must **not block** a
future lineage interface from being added. Concretely:

> "able to send serialized data back from the task execution to Python;
> and an API in the Java SDK for users to be able to specify that data."

### What needs to happen

1. **Reserve a lineage channel on the coordinator IPC.** When the Java
subprocess returns task results to the supervisor, the protocol should
allow an optional serialized lineage payload alongside the existing
result message. The base `BaseCoordinator` interface needs to expose a
hook the supervisor calls with that payload (no-op by default).
2. **Expose a Java SDK API for users to declare lineage data.** Minimal
shape, mirroring how Python tasks can attach lineage to a `TaskInstance`
today. Exact API to be decided once the IPC channel exists, but it
should be:
- opt-in (no overhead for tasks that don't use it),
- language-idiomatic on the Java side,
- resolvable to whatever serialized form the Python listener expects.
3. **Wire the payload into the existing listener pipeline.** The Python
supervisor should forward the serialized lineage data into the same
`get_listener_manager().hook.*` call chain that already runs around
Java task execution, so OL providers don't need any Java-specific code
path.
4. **Document the v1 boundary clearly.** The coordinator user guide should
state that OL start/complete events fire for Java tasks today, but
Java-side lineage extraction is a follow-up.

### Acceptance criteria

- A user enabling OpenLineage sees start/complete events for Java stub
tasks the same way as Python tasks (no regression from current
behavior).
- The coordinator IPC has a documented optional lineage field; a Java
task that does not emit lineage produces exactly the same wire traffic
as today.
- A Java SDK user can attach lineage data from a task and see it land in
the Python supervisor as a serialized payload available to OL listeners.
- The OL provider's listener does not need a Java-specific branch — the
Java payload reaches it through the same listener hooks as Python.
- The coordinator user guide states the v1 boundary and links to this
issue for follow-up work.

### Context

- Dev@ thread:
— Maciej Obuchowski reply on 2026-05-13 ("Generic task information and
specific lineage data ..."). Jarek Potiuk's prior message on the same
day pinged Maciej and Kacper for OL input.
- AIP-108 wiki:
- Originating PRs: apache/airflow#65956 (Java SDK), apache/airflow#65958
(Coordinator layer).
- Related work:
- #66543 — Java-based task and Dag-level callbacks. Callbacks may be
the mechanism (or share machinery) for shipping lineage data back to
Python.
- #66838 — Pluggable communication channels. The IPC reservation
in step 1 should be designed against whatever shape `BaseCoordinator`
settles on there.
- #66590 — Compatible protocol between coordinator and lang-SDK. The
optional lineage field needs to fit the forward-compat contract being
defined there.

[maciej-reply]: https://lists.apache.org/thread/gjot4bxj9kygj2fk76kx6tyg8s4hr057