AlexsJones / AlexsJones/llmfit

GPU not detected on Android (Termux/PRoot) — e.g. Samsung S23 Ultra, Adreno

Ouverte
#175 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
Android documentation Feature Request Hardware Compatibility Improvement Termux
Langage dominant
Rust
Étoiles
36.5k
Forks
2.3k
Merge moyen
3 j 1 h
PR mergées (30 j)
95

Description

## Environment

- **Device:** Samsung S23 Ultra (Qualcomm Snapdragon 8 Gen 2, Adreno 740 GPU)
- **Stack:** Android → Termux → PRoot Ubuntu → running llmfit
- **llmfit:** v0.6.3 (prebuilt `aarch64-unknown-linux-musl` binary)

## Issue

llmfit reports **no GPU detected** on this device. The system has a capable GPU (Adreno 740), but the tool only sees:

- CPU: Cortex-A510 (8 cores)
- Total RAM: 6.91 GB
- Backend: CPU (ARM)
- **GPU: Not detected**

So all model recommendations are CPU-only (“Marginal” fit, no GPU-based suggestions).

## Why this happens

llmfit’s GPU detection is built for **desktop/server** environments:

- **NVIDIA:** `nvidia-smi` (and Linux sysfs fallback)
- **AMD:** `rocm-smi` or Linux DRM/sysfs
- **Intel:** sysfs / `lspci`
- **Apple:** `system_profiler` (macOS)
- **Ascend:** `npu-smi`

On **Android (Termux + PRoot)**:

- None of these interfaces are available. There is no `nvidia-smi`, no `rocm-smi`, and no standard Linux DRM/sysfs GPU nodes visible to the proot environment.
- The GPU is driven by the Android graphics stack (Vulkan/OpenGL ES via the Android HAL), not by the usual Linux GPU APIs that llmfit (or any generic Linux binary) can query.
- From inside PRoot, the system therefore looks like a headless Linux box with no GPU, even though the hardware has one.

So the “no GPU” result is expected with the **current** detection logic; the gap is that **Android / SoC GPUs with unified memory** are not considered.

## Workaround (current behavior)

Using the **`--memory`** override lets users get GPU-style recommendations without detection:

```bash
llmfit --memory=8G fit -n 20
llmfit --memory=8G recommend --json --limit 10
```

That’s a manual workaround, not automatic detection.

## Possible directions (not proposed as full solutions)

Ideas that might help, without claiming they are complete or correct:

1. **Android / unified-memory heuristic**
When on Linux ARM/aarch64 and no GPU has been detected, check for an Android environment (e.g. `ANDROID_ROOT` set, or `/system/build.prop` present, as in Termux/PRoot). If so, treat the system as having a **Vulkan** GPU with **unified memory** (e.g. use total system RAM as the “VRAM” budget). That would align with devices like Snapdragon + Adreno where the GPU shares RAM with the CPU.

2. **`--memory` override backend on ARM Linux**
When the user supplies `--memory` and no GPU is detected, creating a synthetic GPU with **Vulkan** (instead of e.g. Metal) on Linux aarch64 would give more accurate backend/speed assumptions for Android and other ARM Linux environments.

3. **Documentation**
Mention in the docs that on **Android (Termux/PRoot)** GPU detection is not supported and recommend using `--memory=` to get GPU-style recommendations.

4. **Vulkan/DRM probing (if ever visible)**
If in some setups Android or a future environment exposes GPU info via Vulkan or minimal DRM/sysfs, adding a fallback probe there could improve detection without relying only on vendor tools.

I’m opening this issue to document the limitation and the environment, and to suggest the above as possible directions—not as ready-made solutions or a PR.

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.