[Refactoring]: Rebuild Python UDF on Arrow batches and an isolated runtime

## Why do you want to refactor this code?

The current Python UDF path converts MatrixOne vectors into the custom protobuf `DataVector/Data` model value by value, sends those protobuf objects over a bidirectional gRPC stream, materializes Python object lists, and performs the reverse conversion for results.

This boundary has three long-term problems:

- it duplicates a partial SQL type system in the UDF wire protocol and allocates/serializes per value;
- `VECTOR` mode exposes Python lists rather than a standard columnar interface, limiting PyArrow/Pandas/Polars interoperability;
- the current per-CN shared sidecar, mutable package path, and query-time package installation do not provide an adequate revision, isolation, resource-ownership, upgrade, or multi-tenant security contract.

The invariant for the refactor is:

> MatrixOne keeps MO Vector as its native in-engine execution format. Arrow is the versioned columnar interoperability format only at the Python UDF process boundary. Python code must never depend on the private MO Vector ABI.

## Dependency and start gate

**Blocked by #23684. Do not begin production implementation until #23684 is complete and its transport-neutral Arrow primitives and ownership contracts are merged and stable on `main`.**

The intended reusable foundation from #23684 is limited to generic mechanisms such as:

- Arrow IPC structural validation;
- Arrow ArrayData to immutable MO backing adoption/materialization;
- `BufferLease`, per-backing ownership, copy-on-write, accounting, and copy/pin observability.

This issue must not reuse or inherit:

- Arrow `LOAD DATA`'s widening, timezone interpretation, or other ingestion conversion policy;
- the Sirius offload wire protocol, capability model, or execution lease as the Python UDF ABI/runtime;
- static-file scheduling or FileService semantics as an invocation protocol.

Completion of #23684 is a prerequisite, not evidence that this refactor's UDF-specific contracts are implemented.

This is a major cross-subsystem refactor. Before production code starts, the long-term design must be published as a stable versioned document, formally reviewed by Architecture, Security, SQL/Planner, and Cloud/Operator owners, and the exact approved revision must be linked from this issue. Split implementation into reviewed subtasks/PRs after that gate passes.

## Describe the solution you'd like

Build a versioned Python UDF v2 architecture with these boundaries:

1. **Function identity and revisions**
- Resolve an immutable `FunctionRef(function_id, revision)` at planning time.
- Keep code artifacts and Python environments content-addressed by digest.
- Never resolve a mutable latest revision or install packages in the query hot path.

2. **CN UDF operator and Arrow bridge**
- Keep MO Vector inside the engine.
- Add a bounded MO-to-Arrow encoder for inputs and a UDF-specific exact Arrow-to-MO binder for results.
- Preserve SQL NULL, decimal precision/scale, timestamp/timezone, binary, JSON, UUID, charset/collation, and value-domain semantics without truthiness, stringification, widening, truncation, rounding, or timezone guessing.
- Support NULL compact/scatter without changing row order or cardinality.

3. **Versioned control plane and Arrow data plane**
- Use a small versioned control API for capabilities, admission, open/cancel/status, immutable revision/digest identity, budgets, and error classes.
- Use Arrow Flight `DoExchange`/RecordBatch for the CN-to-runtime data plane instead of the per-value protobuf protocol.
- Fence every frame with `invocation_id`, `lease_epoch`, and monotonic `sequence_no`; do not transparently retry an invocation after user code has started.
- Bound both bytes and batch count in flight and propagate cancellation/deadlines end to end.

4. **Trusted runtime boundary and isolated workers**
- Terminate authentication, Flight credentials, admission, resource measurement, Arrow validation, and the PREPARED/Run gate in trusted Gateway/Supervisor components.
- Treat the Python interpreter and its output as untrusted.
- Run user code in resource-bounded, tenant-aware sandboxes; the interpreter must not receive production credentials or direct CN memory access.
- Make every resource owner and every success/error/cancel/timeout/crash/restart cleanup transition explicit and bounded.

5. **Python SDK**
- Let vector handlers consume and return PyArrow arrays/record batches as the primary ABI.
- Keep scalar handlers as a compatibility/convenience API, with row-wise Python object conversion confined to the worker.
- Publish a versioned type/exception/context contract and generated Go/Python conformance fixtures.

6. **Migration and rollout**
- Introduce capability-gated schema shadow, dual resolution/validation, v2 new-write, execution cutover, and eventual v1 retirement stages.
- Keep the existing path available only as an explicit legacy path during the rollback window.
- A runtime outage must fail only queries that require Python UDF; ordinary SQL service remains available.

## Alternatives considered

- **Expose MO Vector directly to Python:** rejected because it leaks MatrixOne's private ABI, ownership model, and release cadence into the Python ecosystem.
- **Only optimize the existing protobuf RPC:** rejected because it retains the duplicate per-value type system and does not solve revision, sandbox, lifecycle, or resource-ownership problems.
- **Embed CPython in CN:** rejected because interpreter/native-extension failures, GIL behavior, and tenant isolation would enter the database process blast radius.
- **Keep one permanent shared sidecar per CN:** acceptable only as a development or deployment adapter, not as the long-term semantic or security boundary.

## Acceptance criteria

- The v2 data path contains no per-row protobuf `Data` encoding; inputs and results use bounded Arrow RecordBatch streams.
- The MO engine remains MO-Vector-native; Python vector handlers use PyArrow without depending on MO's internal ABI.
- A versioned exact type matrix and Go/Python conformance suite covers supported values plus rejection cases for NULLs, overflow, decimal, temporal types, binary, JSON, UUID, malformed offsets/bitmaps/dictionaries/nesting, row-count mismatch, and schema mismatch.
- Deterministic tests cover success, handler error, cancellation, timeout, disconnect, duplicate/out-of-order sequence, stale epoch, worker crash, runtime restart, replace/drop during execution, and cleanup of every owned resource.
- Artifact/environment build and verification are outside the query hot path; published revisions are immutable and digest-verified.
- Sandbox tests cover tenant/revision isolation, filesystem/network/credential restrictions, process/thread/FD/memory/CPU bounds, hostile output, and failed reset/destruction.
- Mixed-version capability gating, upgrade, rollback, backup/restore, runtime drain/readiness, and v1 migration are demonstrated before default cutover.
- Benchmarks report serialization/copy cost, throughput, latency, peak memory, in-flight bytes, scalar/vector behavior, and warm/cold execution; the Arrow path shows no unacceptable regression against the current protobuf baseline.
- The approved design revision and all implementation subtasks/PRs are linked from this issue.