huggingface / huggingface/diffusers

AttributeError: 'str' object has no attribute 'convert' when training controlnet

Aperta
#6,858 17 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

### Describe the bug

when I training controlnet (by **example/controlnet** ), `AttributeError: 'str' object has no attribute 'convert' ` appears

### Reproduction

I use the same bash with example/controlnet/README.md

```
accelerate launch train_controlnet.py \
--pretrained_model_name_or_path=$MODEL_DIR \
--output_dir=$OUTPUT_DIR \
--dataset_name=fusing/fill50k \
--resolution=512 \
--learning_rate=1e-5 \
--validation_image "./conditioning_image_1.png" "./conditioning_image_2.png" \
--validation_prompt "red circle with blue background" "cyan circle with brown floral background" \
--train_batch_size=1 \
--gradient_accumulation_steps=4

```

I download **fill50K** dataset and put it in **example/controlnet**. The folder like this:

```
example/controlnet:
-- train_controlnet.py
-- conditioning_image_1.png
-- conditioning_image_2.png
-- fusing/fill50k
-- train.jsonl
-- conditioning_images
-- 0.png
-- 1.png
......
-- images
-- 0.png
-- 1.png
......
```

### Logs

```shell
Downloading data files: 100%|████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 11066.77it/s]
Extracting data files: 100%|██████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1844.46it/s]
Generating train split: 50000 examples [00:00, 1874549.27 examples/s]
02/05/2024 20:05:51 - INFO - __main__ - ***** Running training *****
02/05/2024 20:05:51 - INFO - __main__ - Num examples = 50000
02/05/2024 20:05:51 - INFO - __main__ - Num batches each epoch = 50000
02/05/2024 20:05:51 - INFO - __main__ - Num Epochs = 1
02/05/2024 20:05:51 - INFO - __main__ - Instantaneous batch size per device = 1
02/05/2024 20:05:51 - INFO - __main__ - Total train batch size (w. parallel, distributed & accumulation) = 4
02/05/2024 20:05:51 - INFO - __main__ - Gradient Accumulation steps = 4
02/05/2024 20:05:51 - INFO - __main__ - Total optimization steps = 12500
Steps: 0%| | 0/12500 [00:00
main(args)
File "/mnt/sdb/数据2/diffusers/examples/controlnet/train_controlnet.py", line 1007, in main
for step, batch in enumerate(train_dataloader):
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/accelerate/data_loader.py", line 384, in __iter__
current_batch = next(dataloader_iter)
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 633, in __next__
data = self._next_data()
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 677, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = self.dataset.__getitems__(possibly_batched_index)
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 2799, in __getitems__
batch = self.__getitem__(keys)
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 2795, in __getitem__
return self._getitem(key)
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 2780, in _getitem
formatted_output = format_table(
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 629, in format_table
return formatter(pa_table, query_type=query_type)
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 400, in __call__
return self.format_batch(pa_table)
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 515, in format_batch
return self.transform(batch)
File "/mnt/sdb/diffusers/examples/controlnet/train_controlnet.py", line 681, in preprocess_train
images = [image.convert("RGB") for image in examples[image_column]]
File "/mnt/sdb/diffusers/examples/controlnet/train_controlnet.py", line 681, in
images = [image.convert("RGB") for image in examples[image_column]]
AttributeError: 'str' object has no attribute 'convert'
Steps: 0%| | 0/12500 [00:00
sys.exit(main())
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 47, in main
args.func(args)
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/accelerate/commands/launch.py", line 986, in launch_command
simple_launcher(args)
File "/home/ps/anaconda3/envs/diffusers/lib/python3.10/site-packages/accelerate/commands/launch.py", line 628, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/home/ps/anaconda3/envs/diffusers/bin/python3.10', 'train_controlnet.py', '--pretrained_model_name_or_path=/home/ps/train_sd_code/diffusers/runwayml/stable-diffusion-v1-5', '--output_dir=./output_test1', '--dataset_name=fusing/fill50k', '--resolution=512', '--learning_rate=1e-5', '--validation_image', './conditioning_image_1.png', './conditioning_image_2.png', '--validation_prompt', 'red circle with blue background', 'cyan circle with brown floral background', '--train_batch_size=1', '--gradient_accumulation_steps=4']' returned non-zero exit status 1.
```

### System Info

I tried diffusers-0.25.0.dev0 and diffusers-0.26.0.dev0, but both have the same problem

### Who can help?

@sayakpaul @yiyixuxu @DN6 @patrickvonplaten

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start in examples/controlnet/train_controlnet.py at preprocess_train, especially line 681, and compare the fill50k dataset columns and values with the README command. Reproduce the failure with the provided accelerate launch command and inspect how the dataset is formatted before preprocessing. Done means training can begin without the AttributeError.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
machine-learning
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Attiva
Chiarezza
Abbastanza chiara
Idoneità per principianti
65/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.