pytorch / pytorch/executorch

[RFC][Android] Consolidate LlmModule constructor and generate() overloads

Open
#17,637 2 comments 1 reaction 1 assignee View on GitHub

@mergennachin is already working on this.

Since Feb 23, 2026.

module: android
Dominant language
Python
Stars
5k
Forks
1.2k
Avg merge
2d 10h
Merged PRs (30d)
581

Description

🐛 Describe the bug

Problem

LlmModule has 8 constructors and 10 generate() overloads. A codebase-wide search (executorch + executorch-examples) shows most are unused.

Constructors — 4 of 8 have zero call sites
Constructor Call sites
(String, String, float) 3 (benchmark, instrumentation test, sanity check)
(int, String, String, float) 1 (LlamaDemo, no dataPath)
(int, String, String, float, String) 1 (LlamaDemo, with dataPath)
(int, String, String, float, List<String>) 2 (LlamaDemo LoRA)
(int, String, String, float, List<String>, int, int) 0 (only internal delegate target)
(int, String, String, float, String, int, int) 0
(String, String, float, String) 0
(LlmModuleConfig) 0
generate() — 7 of 10 have zero call sites
Overload Call sites
(String, LlmCallback) 2
(String, int, LlmCallback) 2
(String, int, LlmCallback, boolean) 4
(String, LlmCallback, boolean) 0
(String, int, LlmCallback, boolean, float, int, int) 0 (native JNI entry, only internal)
(String, LlmGenerationConfig, LlmCallback) 0
All 4 image (int[], int, int, int, String, ...) variants 0 (image input now goes through prefillImages() + text-only generate())

LlmModuleConfig and LlmGenerationConfig builder classes exist but neither is used anywhere — all callers use the direct constructors/overloads instead.

Proposal

Phase 1: Deprecate and redirect
  1. Keep one constructor: LlmModule(LlmModuleConfig). Deprecate all others with @Deprecated pointing to the config builder.

  2. Keep two generate() signatures:

    • generate(String prompt, LlmGenerationConfig config, LlmCallback callback) — the config-based entry point
    • generate(String prompt, int seqLen, LlmCallback callback, boolean echo, float temperature, int numBos, int numEos) — the native JNI method (package-private or private)

    Deprecate all other generate() overloads.

  3. Remove all image-based generate() overloads — they are dead code. Multimodal input already uses the prefillImages()/prefillAudio()/prefillPrompt()generate() flow.

  4. Fix LlmModuleConfig.getDataPath() to return List<String> instead of String, matching what the constructor actually needs.

  5. Deduplicate model type constants — remove MODEL_TYPE_* from LlmModule, keep only in LlmModuleConfig.

Phase 2: Remove (next release)

Remove all deprecated overloads.

Migration

All existing call sites migrate to:

// Before
LlmModule module = new LlmModule(MODEL_TYPE_TEXT, modelPath, tokenizerPath, 0.8f);
module.generate(prompt, seqLen, callback, false);

// After
LlmModule module = new LlmModule(
    LlmModuleConfig.create()
        .modulePath(modelPath)
        .tokenizerPath(tokenizerPath)
        .temperature(0.8f)
        .build());
module.generate(prompt, LlmGenerationConfig.create()
    .seqLen(seqLen)
    .echo(false)
    .build(), callback);

Compatibility

LlmModule is annotated @Experimental ("subject to change without notice"), so breaking changes are acceptable. A deprecation phase is still recommended since external apps may use these APIs.

Scope

Java-only change. No JNI or C++ changes needed — the single native generate() method stays as-is.

Versions

0d9799f

0d9799f

cc @cbilgin

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.