jd-opensource / jd-opensource/JoyAI-Image

Batch inference support

Open
#5 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.2k
Forks
165
PR merge metrics
No merged PRs in 30d

Description

RTX PRO 6000
59gb vram usage per inference call
90 seconds per output image

```
python3 '/root/instance/JoyAI-Image/inference.py' --ckpt-root '/root/instance/models/JoyAI_Image_Edit' --prompt 'Move the camera.
- Camera rotation: Yaw 180°, Pitch 30°.
- Camera zoom: unchanged.
- Keep the 3D scene static; only change the viewpoint.' --image '/root/instance/in/kik_test.png' --output '/root/instance/out/kik/a.png' --seed '8954' --steps '50' --guidance-scale '4' --neg-prompt '' --basesize '1024' --hsdp-shard-dim '1'
- ```

Currently the model loads and unloads to vram on every call.
I would like to ask if its possible to implement batch mode.

--prompts_file '/root/instance/in/prompts.json'

Maybe some other tricks to speed it up?

Thanks.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.