intel / intel/AI-Playground

AI Playground fails to load Gemma‑3‑12B‑IT both remotely and locally due to incorrect Hugging Face validation

Open
#419 1 comment 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
TypeScript
Stars
979
Forks
132
Avg merge
9h 23m
Merged PRs (30d)
8

Description

## Describe the bug

There are two separate but related issues preventing Gemma‑3‑12B‑IT GGUF from loading in AI Playground (v3.0.2‑beta):

Bug 1 — Remote loading fails
AI Playground cannot load the model from Hugging Face because the GGUF files are stored inside a /gguf/ subfolder.

This causes:

“Please provide a valid model reference”
Attempt to download the entire repo (e.g., 186 GB)
“Not enough disk space (undefined)”
Failure to detect .gguf files in nested folders

Bug 2 — Local loading fails

Even after manually placing the .gguf file in the local models folder, selecting it triggers a Hugging Face download popup.

The backend:
Treats the filename as a Hugging Face repo ID
Fails with 401 / NotImplementedError
Never checks the local filesystem
Leaves the Confirm button disabled
Blocks inference entirely

Both issues together make the model unusable.

## To Reproduce

Bug 1 — Remote loading fails (subfolder repo)

Go to Chat Settings → Add Model → Hugging Face

Enter:
unsloth/gemma-3-12b-it-GGUF

AI Playground attempts to download the entire repository

Shows:
“Not enough disk space. It requires undefined…”

Cause
The GGUF files are inside a /gguf/ subfolder.
AI Playground cannot detect files in nested folders and assumes the repo is empty.

Bug 2 — Local loading fails (incorrect Hugging Face validation)

Steps to reproduce

Download gemma-3-12b-it-Q4_K_M.gguf manually
Place it in Documents/AI-Playground/models/
Restart AI Playground
Select the model in Chat Settings
Enter any prompt
A Hugging Face download popup appears
Confirm button is disabled
Inference fails

Cause
The backend incorrectly tries to validate the local file by querying:

Code
https://huggingface.co/api/models/gemma-3-12b-it-Q4_K_M.gguf

This is not a repo, so Hugging Face returns:
401 Unauthorized
Repository Not Found

Then the backend attempts to list repo contents and crashes:

Code
NotImplementedError: Access to repositories lists is not implemented.

Because of this crash, the backend never checks the local filesystem.

## Expected behavior

For remote Hugging Face models (Bug 1)

AI Playground should correctly detect .gguf files even when they are stored inside subfolders (e.g., /gguf/).

The app should list available GGUF files without attempting to download the entire repository.

Disk‑space requirements should be calculated accurately (not “undefined”).

The user should be able to select a specific .gguf file from the repo and download only that file.

For local models (Bug 2)

If a .gguf file exists in the local models directory, AI Playground should load it immediately, without performing any Hugging Face validation.

The backend should not treat a local filename as a Hugging Face repo ID.

No download popup should appear when the model is already present locally.

The model type and size should be detected correctly (e.g., “GGUF”, “7.3 GB”).

Inference should begin normally once the model is selected.

Overall expected behavior

Local and remote models should be handled through separate, predictable code paths.

The presence of a local model should always override remote checks.

The user should be able to load and run a local .gguf model without errors, popups, or disabled buttons.

## Screenshots

Screenshot A — “Download model” popup

Shows the popup that appears even though the model is local, with:

Type = Undefined

Size blank

Confirm button disabled

Screenshot B — Chat Settings → Model selection
Shows the model appearing in the dropdown list under Chat Settings, confirming the file is detected locally.

## Environment (please complete the following information):

Windows 11
Intel Arc A750
Intel i3‑10100
20 GB RAM
AI Playground v3.0.2‑beta

## Additional context
Error Log Excerpt
GET https://huggingface.co/api/models/gemma-3-12b-it-Q4_K_M.gguf "401 Unauthorized"
Repository Not Found
NotImplementedError: Access to repositories lists is not implemented.
500 INTERNAL SERVER ERROR on /api/isAccessGranted
TypeError: Cannot read properties of undefined

Bug #1 matches the behavior described in Issue #139

Bug #2 appears to be new:

Local models still trigger Hugging Face validation
Backend crashes before checking local files
Confirm button remains disabled

**

Image

Image

**

**For Bug 1 — Remote loading fails (subfolder repo)**

Problem:
AI Playground assumes all GGUF files are located at the root of a Hugging Face repo.
Gemma‑3‑12B‑IT stores its GGUF files inside /gguf/, causing the app to think the repo is empty.

Suggested fix:

Add support for recursively scanning subfolders in Hugging Face repos.

Or allow users to specify a full file path, e.g.:
unsloth/gemma-3-12b-it-GGUF/gguf/gemma-3-12b-it-Q4_K_M.gguf

Or treat .gguf files as discoverable anywhere in the repo tree.

This would resolve the “undefined disk space” and “download entire repo” issues.

**For Bug 2 — Local loading fails (backend incorrectly validates local files)**

Problem:
When a user selects a local .gguf file, the backend still attempts to:

Query Hugging Face using the filename as a repo ID

Check gating status

List repo contents

Determine remote file size

These steps fail and prevent the backend from ever checking the local filesystem.

Suggested fix:

Add a simple early‑exit condition:
If the model exists in the local models directory, skip all Hugging Face validation.

Only perform Hugging Face checks when the user explicitly adds a remote model.

Ensure isAccessGranted, getModelSize, and populate_file_list do not run for local files.

Avoid treating filenames ending in .gguf as Hugging Face repo IDs.

This would allow local models to load even when remote validation fails.

General improvement suggestion
Introduce a clear distinction between:

Local models (no remote checks needed)

Remote Hugging Face models (full validation required)

This separation would prevent both bugs from occurring and improve reliability for all GGUF models.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.