kohya-ss / kohya-ss/sd-scripts
[Bug]: AttributeError: module 'library.train_util' has no attribute 'load_tokenizer' in v25.2.1
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
### Description
I am encountering a crash when attempting to run train_control_net.py. The script fails early during the initialization phase while trying to load the tokenizer. It seems there is a mismatch or a missing attribute in the library.train_util module.
### Command
PowerShell:
`python ./sd-scripts/train_control_net.py --config_file F:\CJY\deep-learning\newkohya_ss\setdata\medical_tile.toml`
### Error Log
> (kohyass) F:\CJY\deep-learning\newkohya_ss>python ./sd-scripts/train_control_net.py --config_file F:\CJY\deep-learning\newkohya_ss\setdata\medical_tile.toml
。。。。。。
Traceback (most recent call last):
File "F:\CJY\deep-learning\newkohya_ss\sd-scripts\train_control_net.py", line 669, in
train(args)
File "F:\CJY\deep-learning\newkohya_ss\sd-scripts\train_control_net.py", line 75, in train
tokenizer = train_util.load_tokenizer(args)
AttributeError: module 'library.train_util' has no attribute 'load_tokenizer'
Traceback (most recent call last):
File "F:\CJY\deep-learning\newkohya_ss\sd-scripts\train_control_net.py", line 669, in
train(args)
File "F:\CJY\deep-learning\newkohya_ss\sd-scripts\train_control_net.py", line 75, in train
tokenizer = train_util.load_tokenizer(args)
AttributeError: module 'library.train_util' has no attribute 'load_tokenizer'
### Configuration (medical_tile.toml)
> [model_arguments]
> pretrained_model_name_or_path = "F:/CJY/deep-learning/sd_img2img_lora/stable-diffusion-v1-5"
> save_model_as = "safetensors"
>
> [dataset_arguments]
> train_data_dir = "F:/CJY/deep-learning/kohya_ss/SET/ct_dataset_10242/target"
> conditioning_data_dir = "F:/CJY/deep-learning/kohya_ss/SET/ct_dataset_10242/source"
> resolution = "512,512"
> enable_bucket = false
> cache_latents = true
> cache_latents_to_disk = false
>
> [training_arguments]
> output_dir = "F:/CJY/deep-learning/newkohya_ss/setdata/train_output"
> logging_dir = "F:/CJY/deep-learning/newkohya_ss/setdata/logs"
> output_name = "contrast_enhancer_v1"
> save_precision = "bf16"
> mixed_precision = "bf16"
> train_batch_size = 1
> gradient_accumulation_steps = 4
> max_train_steps = 15000
> learning_rate = 1e-5
> optimizer_type = "AdamW8bit"
> lr_scheduler = "constant_with_warmup"
> lr_warmup_steps = 500
> gradient_checkpointing = true
> xformers = true
> mem_eff_attn = true
> caption_prefix = "high contrast grayscale, medical imaging, clear boundaries"
> loss_type = "l1"
> min_snr_gamma = 5
> log_with = "tensorboard"
>
> sample_every_n_steps = 500
> sample_prompts = "F:/dataset/sample_prompt.txt"
> sample_sampler = "euler_a"
### Environment
- Repo Version: v25.2.1 (Commit: 4161d1d80ad554f7801c584632665d6825994062)
- OS: Windows
- Python: 3.10 (Assumed from typical kohya_ss env)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with sd-scripts/train_control_net.py at the tokenizer call around line 75, then inspect library.train_util in commit 4161d1d80ad554f7801c584632665d6825994062. Reproduce the Windows command with the supplied configuration and trace why load_tokenizer is unavailable; done means train_control_net.py completes tokenizer initialization without this AttributeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100