lllyasviel / lllyasviel/stable-diffusion-webui-forge

[Bug]: Update transformers dependency version to make tokenizer compilable with stable rustc

Open
#761 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Checklist

- [X] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

`transformers==4.30.2` requires `tokenizers>=0.11.1, <0.11.3 | >0.11.3, <0.14`,
tokenizers needs to be compiled locally on Linux but newest compatible version 0.13.2 can't be compiled using rustc 1.78.0, for some breaking change in 1.6x. [as described here](https://github.com/huggingface/tokenizers/issues/1402#issuecomment-1839004693)

### Steps to reproduce the problem

update transformers version spec.

### What should have happened?

Correctly install.

### What browsers do you use to access the UI ?

Mozilla Firefox, Google Chrome

### Sysinfo

nothing with sysinfo

### Console logs

```Shell
just build failed.
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Locate the dependency specification for transformers, then reproduce the Linux installation or run the reported `just build` command with stable rustc. Done means the compatible dependency set installs successfully without the tokenizers compilation failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.