huggingface / huggingface/local-gemma

Warnings still printed with --silent

Open
#42 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
383
Forks
31
PR merge metrics
No merged PRs in 30d

Description

Running local-gemma with --silent two warnings are printed:

- [First one is probably from here](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/utils.py#L1757-L1758)
- [Second one probably here](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/utils.py#L2415-L2417):

```console
$ local-gemma --silent --model="27b" "What is the capital of Germany?"
An assistant model is provided, using a dynamic cache instead of a cache of type='hybrid'.
From v4.47 onwards, when a model cache is to be returned, `generate` will return a `Cache` instance instead by default (as opposed to the legacy tuple of tuples format). If you want to keep returning the legacy format, please set `return_legacy_cache=True`.
The capital of Germany is **Berlin**.
```

Installed with pipx, package versions as follows:

```console
$ pipx runpip local-gemma freeze
accelerate==1.1.1
bitsandbytes==0.44.1
certifi==2024.8.30
charset-normalizer==3.4.0
filelock==3.16.1
fsspec==2024.10.0
huggingface-hub==0.26.2
idna==3.10
Jinja2==3.1.4
local_gemma==0.2.0
MarkupSafe==3.0.2
mpmath==1.3.0
networkx==3.4.2
numpy==2.1.3
nvidia-cublas-cu12==12.4.5.8
nvidia-cuda-cupti-cu12==12.4.127
nvidia-cuda-nvrtc-cu12==12.4.127
nvidia-cuda-runtime-cu12==12.4.127
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.2.1.3
nvidia-curand-cu12==10.3.5.147
nvidia-cusolver-cu12==11.6.1.9
nvidia-cusparse-cu12==12.3.1.170
nvidia-nccl-cu12==2.21.5
nvidia-nvjitlink-cu12==12.4.127
nvidia-nvtx-cu12==12.4.127
packaging==24.2
psutil==6.1.0
PyYAML==6.0.2
regex==2024.11.6
requests==2.32.3
safetensors==0.4.5
sympy==1.13.1
tokenizers==0.20.3
torch==2.5.1
tqdm==4.67.1
transformers==4.46.3
triton==3.1.0
typing_extensions==4.12.2
urllib3==2.2.3
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the issue with the local-gemma --silent command shown, then trace how the CLI handles silent output alongside the warning locations in transformers/src/transformers/generation/utils.py at lines 1757-1758 and 2415-2417. Done means the command prints the generated answer without either warning.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
cli, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.