huggingface / huggingface/local-gemma

TypeError: 'QuantoConfig' object is not subscriptable

Open
#23 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
383
Forks
31
PR merge metrics
No merged PRs in 30d

Description

**While running the example code in Readme.md**
`from local_gemma import LocalGemma2ForCausalLM
from transformers import AutoTokenizer
import os

os.environ['HUGGINGFACEHUB_API_TOKEN'] = ''
os.environ['HF_TOKEN'] = ''

model = LocalGemma2ForCausalLM.from_pretrained("google/gemma-2-9b",token='')
tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-9b")

model_inputs = tokenizer("The cat sat on the mat", return_attention_mask=True, return_tensors="pt")
generated_ids = model.generate(**model_inputs.to(model.device))

decoded_text = tokenizer.batch_decode(generated_ids)`

Kindly help

Note: I've installed quanto explicitly when prompted in a previous error

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the failure using the README.md example with LocalGemma2ForCausalLM, AutoTokenizer, and the google/gemma-2-9b model. Trace the TypeError involving QuantoConfig and verify the example completes model loading and text generation without the reported exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.