software-mansion / software-mansion/react-native-executorch

Upstream the Gemma 4 Vulkan support we carry on our ExecuTorch fork

Open
#1,361 0 comments 1 reaction 1 assignee View on GitHub

@msluszniak is already working on this.

Since Aug 14, 2026.

performance platform: android
Dominant language
TypeScript
Stars
1.7k
Forks
96
Avg merge
20h 51m
Merged PRs (30d)
59

Description

Our ExecuTorch fork (software-mansion-labs/executorch) carries Gemma 4 Vulkan
support that upstream does not have. It should go upstream so we stop carrying
it, but it is a re-port rather than a patch, so it needs planning.

Context

Commit 24f0f69 "Add Gemma4 Vulkan support (Android GPU)" is 45 files,
+4247/-1188. It was already a port ("Ports Mateusz Kopciński's Gemma4 Vulkan
work onto 1.3"), and the upstream files it rewrites have moved again since
1.3.1. Upstream now has its own RMSNorm.cpp and IndexSelect.cpp, and its
SDPA.cpp has evolved independently, so our version collides rather than
applies.

Everything else SWM carries has now been checked against upstream and is either
already there or has been sent:

  • MLX getitem boundary fix (1f9e5a4) — already upstream, verbatim.
  • MLX iOS support (d454314) — upstream built its own (iOS 17.0+,
    ios.toolchain.cmake, metallib via MLX_METAL_JIT). Superseded.
  • Vulkan SymInt as a plain Int EValue — extracted from the Gemma 4 commit and
    sent as pytorch/executorch#21848.
  • Android per-backend shared libraries — pytorch/executorch#21849.
  • Vulkan index.Tensor support check + delegate arg-count guard —
    pytorch/executorch#21847 (found while enabling LFM2.5-VL, see #1360).

So Gemma 4 Vulkan is the only substantial piece left on the fork.

Suggested split

Four independent pieces, easiest first:

  1. Shader precision/perf helperslinear_q4gsw / linear_dq8ca_q4gsw
    changes plus sdpa_fp32_acc.glslh and linear_fp32_scale_dq_acc.glslh.
    Additive, likely stand alone.
  2. The aten.rms_norm lowering (replacing the legacy et_vk.rms_norm).
    Upstream already has an RMSNorm.cpp; this needs a diff against theirs first
    to establish whether ours is an improvement, a duplicate, or a regression.
  3. GemmaSDPA.cpp + the gemma SDPA shaders — the actual feature, ~738 lines
    plus ~10 shader files. New files, so it can land as an addition, but it
    depends on the reworked SDPA tile-load helpers, which is exactly where it
    collides with upstream's evolved SDPA.cpp. This is the bulk of the work.
  4. Do not upstream — the gen_vulkan_spv.py backward-compat shim (lets
    ported shaders call define_required_extensions with one arg; a porting
    artifact) and the DispatchNode.cpp tracer, see below.

Separate cleanup, worth doing regardless

The same commit added an env-gated dispatch tracer to
backends/vulkan/runtime/graph/ops/DispatchNode.cppETVK_DISPATCH_TRACE,
with <cstdio>/fopen/setvbuf, commented "Iter 21 instrumentation". It is
still in the fork HEAD, so it ships in every runtime we build, including the one
RNE distributes. Inert unless the env var is set, but it is leftover debugging
in a shipped runtime and should probably come out of the fork.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.