Inference time

Hi,

Thanks a lot for sharing this amazing work. I'm quite new to LLM and VLM, and I tried running your 'InternLM-XComposer-2.5' model. I followed the provided 'InternLM-XComposer-2.5 with Transformers, Multi-Image Multi-Tune Dialog' code exactly, using the example with 4 car images. However, it took a very long time to get the inference results.

My system specs are: 128GB RAM, Nvidia GeForce RTX 3090 GPU with 24GB VRAM. I am running the code on WSL. I noticed that my GPU's dedicated memory was almost fully utilized, and it was using nearly 50GB of shared GPU memory. Additionally, my RAM usage was around 30-50GB.

Given my system specifications, I'm wondering if it's normal for inference to take this long. The Hugging Face demo you provided outputs results much faster. Could you share the specifications used for that demo?

Thanks!