pytorch / pytorch/benchmark

install_diffusers issue with stable_diffusion_xl

Open
#2,522 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1k
Forks
346
PR merge metrics
No merged PRs in 30d

Description

The call to install_diffusers() in benchmark/torchbenchmark/canary_models/stable_diffusion_xl/install.py at main · pytorch/benchmark should be made before error checking for the token, similarly to how it's done for other models such as stable_diffusion_unet and stable_diffusion_text_encoder. If not, then when it warns about the token and exits, it has not installed diffusers yet.

The following is an example of the error when we try to run the model after installation and the token is not set.

$ python3 run.py stable_diffusion_xl -t eval
Warning: The model stable_diffusion_xl cannot be found at core set.
Traceback (most recent call last):
  File "/home/user/benchmark/run.py", line 599, in <module>
    main()  # pragma: no cover
  File "/home/user/benchmark/run.py", line 511, in main
    model_flops = get_model_flops(config) if "model_flops" in metrics_needed else None
  File "/home/user/benchmark/torchbenchmark/util/experiment/metrics.py", line 129, in get_model_flops
    model = load_model(eager_model_config)
  File "/home/user/benchmark/torchbenchmark/util/experiment/instantiator.py", line 90, in load_model
    Model = load_canary_model_by_name(config.name)
  File "/home/user/benchmark/torchbenchmark/__init__.py", line 686, in load_canary_model_by_name
    module = importlib.import_module(f".canary_models.{model}", package=__name__)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/user/benchmark/torchbenchmark/canary_models/stable_diffusion_xl/__init__.py", line 11, in <module>
    from diffusers import DiffusionPipeline
ModuleNotFoundError: No module named 'diffusers'

Note that this is a low priority issue and a bit of an edge case as it doesn’t affect performance if the token is set.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read torchbenchmark/canary_models/stable_diffusion_xl/install.py and compare its install_diffusers() ordering with the stable_diffusion_unet and stable_diffusion_text_encoder install.py files. Run python3 run.py stable_diffusion_xl -t eval without a token to verify that diffusers is installed before the token warning exits.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.