kohya-ss / kohya-ss/sd-scripts

[Bug]: AttributeError: module 'library.train_util' has no attribute 'load_tokenizer' in v25.2.1

Open
#2,249 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

### Description
I am encountering a crash when attempting to run train_control_net.py. The script fails early during the initialization phase while trying to load the tokenizer. It seems there is a mismatch or a missing attribute in the library.train_util module.

### Command
PowerShell:

`python ./sd-scripts/train_control_net.py --config_file F:\CJY\deep-learning\newkohya_ss\setdata\medical_tile.toml`
### Error Log
> (kohyass) F:\CJY\deep-learning\newkohya_ss>python ./sd-scripts/train_control_net.py --config_file F:\CJY\deep-learning\newkohya_ss\setdata\medical_tile.toml
。。。。。。
Traceback (most recent call last):
File "F:\CJY\deep-learning\newkohya_ss\sd-scripts\train_control_net.py", line 669, in
train(args)
File "F:\CJY\deep-learning\newkohya_ss\sd-scripts\train_control_net.py", line 75, in train
tokenizer = train_util.load_tokenizer(args)
AttributeError: module 'library.train_util' has no attribute 'load_tokenizer'

Traceback (most recent call last):
File "F:\CJY\deep-learning\newkohya_ss\sd-scripts\train_control_net.py", line 669, in
train(args)
File "F:\CJY\deep-learning\newkohya_ss\sd-scripts\train_control_net.py", line 75, in train
tokenizer = train_util.load_tokenizer(args)
AttributeError: module 'library.train_util' has no attribute 'load_tokenizer'

### Configuration (medical_tile.toml)

> [model_arguments]
> pretrained_model_name_or_path = "F:/CJY/deep-learning/sd_img2img_lora/stable-diffusion-v1-5"
> save_model_as = "safetensors"
>
> [dataset_arguments]
> train_data_dir = "F:/CJY/deep-learning/kohya_ss/SET/ct_dataset_10242/target"
> conditioning_data_dir = "F:/CJY/deep-learning/kohya_ss/SET/ct_dataset_10242/source"
> resolution = "512,512"
> enable_bucket = false
> cache_latents = true
> cache_latents_to_disk = false
>
> [training_arguments]
> output_dir = "F:/CJY/deep-learning/newkohya_ss/setdata/train_output"
> logging_dir = "F:/CJY/deep-learning/newkohya_ss/setdata/logs"
> output_name = "contrast_enhancer_v1"
> save_precision = "bf16"
> mixed_precision = "bf16"
> train_batch_size = 1
> gradient_accumulation_steps = 4
> max_train_steps = 15000
> learning_rate = 1e-5
> optimizer_type = "AdamW8bit"
> lr_scheduler = "constant_with_warmup"
> lr_warmup_steps = 500
> gradient_checkpointing = true
> xformers = true
> mem_eff_attn = true
> caption_prefix = "high contrast grayscale, medical imaging, clear boundaries"
> loss_type = "l1"
> min_snr_gamma = 5
> log_with = "tensorboard"
>
> sample_every_n_steps = 500
> sample_prompts = "F:/dataset/sample_prompt.txt"
> sample_sampler = "euler_a"

### Environment

- Repo Version: v25.2.1 (Commit: 4161d1d80ad554f7801c584632665d6825994062)
- OS: Windows
- Python: 3.10 (Assumed from typical kohya_ss env)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with sd-scripts/train_control_net.py at the tokenizer call around line 75, then inspect library.train_util in commit 4161d1d80ad554f7801c584632665d6825994062. Reproduce the Windows command with the supplied configuration and trace why load_tokenizer is unavailable; done means train_control_net.py completes tokenizer initialization without this AttributeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.