deepseek-ai / deepseek-ai/DreamCraft3D
Unclear installation steps + run error
- Dominant language
- Python
- Stars
- 3k
- Forks
- 357
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I have followed the steps provided and installed threestudio, which went fine.
But then, it is unclear if I have to git clone this repo and run from inside that directory, which results in missing yaml-files, or should overwrite the files in the threestudio directory with this repo's files, because when I only get clone and then run, the load-directory is missing, so yeah. :-/
My command is:
`python launch.py --config configs/dreamcraft3d-coarse-nerf.yaml --train system.prompt_processor.prompt="young woman, scarf, white marble, bust, statue, victorian" data.image_path="load/images/Z1.png"`
I tried with your repo and copied the load-directory from threestudio. This resulted in this error:
```
Seed set to 0
[INFO] Loading Deep Floyd ...
unet\diffusion_pytorch_model.fp16.safetensors not found
A mixture of fp16 and non-fp16 filenames will be loaded.
Loaded fp16 filenames:
[text_encoder/pytorch_model.fp16-00002-of-00002.bin, safety_checker/pytorch_model.fp16.bin, text_encoder/pytorch_model.fp16-00001-of-00002.bin, unet/diffusion_pytorch_model.fp16.bin]
Loaded non-fp16 filenames:
[watermarker/diffusion_pytorch_model.bin
If this behavior is not expected, please check your folder structure.
Loading pipeline components...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:07<00:00, 2.53s/it]
[INFO] Loaded Deep Floyd!
[INFO] Loading Stable Zero123 ...
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.53 M params.
Keeping EMAs of 688.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Traceback (most recent call last):
File "C:\Users\Andy\Documents\DreamCraft3D\launch.py", line 252, in
main(args, extras)
File "C:\Users\Andy\Documents\DreamCraft3D\launch.py", line 120, in main
system: BaseSystem = threestudio.find(cfg.system_type)(
File "C:\Users\Andy\Documents\DreamCraft3D\threestudio\systems\base.py", line 45, in __init__
self.configure()
File "C:\Users\Andy\Documents\DreamCraft3D\threestudio\systems\dreamcraft3d.py", line 45, in configure
self.guidance_3d = threestudio.find(self.cfg.guidance_3d_type)(
File "C:\Users\Andy\Documents\DreamCraft3D\threestudio\utils\base.py", line 83, in __init__
self.configure(*args, **kwargs)
File "C:\Users\Andy\Documents\DreamCraft3D\threestudio\models\guidance\stable_zero123_guidance.py", line 137, in configure
self.prepare_embeddings(self.cfg.cond_image_path)
File "C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\site-packages\torch\amp\autocast_mode.py", line 16, in decorate_autocast
return func(*args, **kwargs)
File "C:\Users\Andy\Documents\DreamCraft3D\threestudio\models\guidance\stable_zero123_guidance.py", line 149, in prepare_embeddings
assert os.path.exists(image_path)
AssertionError
```
If I go for a version in which I overwrite the original threestudio with your repo's files, I get this error instead:
```
Seed set to 0
[INFO] Loading Deep Floyd ...
unet\diffusion_pytorch_model.fp16.safetensors not found
A mixture of fp16 and non-fp16 filenames will be loaded.
Loaded fp16 filenames:
[text_encoder/pytorch_model.fp16-00001-of-00002.bin, safety_checker/pytorch_model.fp16.bin, text_encoder/pytorch_model.fp16-00002-of-00002.bin, unet/diffusion_pytorch_model.fp16.bin]
Loaded non-fp16 filenames:
[watermarker/diffusion_pytorch_model.bin
If this behavior is not expected, please check your folder structure.
Loading pipeline components...: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:07<00:00, 2.53s/it]
[INFO] Loaded Deep Floyd!
[INFO] Loading Stable Zero123 ...
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.53 M params.
Keeping EMAs of 688.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
[INFO] Loaded Stable Zero123!
[INFO] Using prompt [young woman, scarf, white marble, bust, statue, victorian] and negative prompt []
[INFO] Using view-dependent prompts [side]:[young woman, scarf, white marble, bust, statue, victorian, side view] [front]:[young woman, scarf, white marble, bust, statue, victorian, front view] [back]:[young woman, scarf, white marble, bust, statue, victorian, back view] [overhead]:[young woman, scarf, white marble, bust, statue, victorian, overhead view]
C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_5m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_11m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_11m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_384 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_384. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_512 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_512. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
Process SpawnProcess-1:
Traceback (most recent call last):
File "C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\multiprocessing\process.py", line 314, in _bootstrap
self.run()
File "C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\Andy\Documents\threestudio\threestudio\models\prompt_processors\deepfloyd_prompt_processor.py", line 63, in spawn_func
text_encoder = T5EncoderModel.from_pretrained(
File "C:\Users\Andy\Anaconda3\envs\dreamcraft3d\lib\site-packages\transformers\modeling_utils.py", line 2505, in from_pretrained
raise EnvironmentError(
OSError: DeepFloyd/IF-I-XL-v1.0 does not appear to have a file named pytorch_model.8bit.bin but there is a file without the variant 8bit. Use `variant=None` to load this model from those weights.
Traceback (most recent call last):
File "C:\Users\Andy\Documents\threestudio\launch.py", line 252, in
main(args, extras)
File "C:\Users\Andy\Documents\threestudio\launch.py", line 120, in main
system: BaseSystem = threestudio.find(cfg.system_type)(
File "C:\Users\Andy\Documents\threestudio\threestudio\systems\base.py", line 45, in __init__
self.configure()
File "C:\Users\Andy\Documents\threestudio\threestudio\systems\dreamcraft3d.py", line 50, in configure
self.prompt_processor = threestudio.find(self.cfg.prompt_processor_type)(
File "C:\Users\Andy\Documents\threestudio\threestudio\utils\base.py", line 83, in __init__
self.configure(*args, **kwargs)
File "C:\Users\Andy\Documents\threestudio\threestudio\models\prompt_processors\base.py", line 337, in configure
self.load_text_embeddings()
File "C:\Users\Andy\Documents\threestudio\threestudio\models\prompt_processors\base.py", line 395, in load_text_embeddings
self.text_embeddings = self.load_from_cache(self.prompt)[None, ...]
File "C:\Users\Andy\Documents\threestudio\threestudio\models\prompt_processors\base.py", line 413, in load_from_cache
raise FileNotFoundError(
FileNotFoundError: Text embedding file .threestudio_cache/text_embeddings\aca2a23effdf0d18401b69d88fd03720.pt for model DeepFloyd/IF-I-XL-v1.0 and prompt [young woman, scarf, white marble, bust, statue, victorian] not found.
```
What is the right way? And could you add that to the installation guide please?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.