Difference in Response Quality Between Hosted Demo and Local Molmo-7B-D

Hi there!

I’ve noticed that the quality of responses generated by the Molmo-7B-D model seems to differ between the hosted demo and when I run it locally. The hosted demo responses appear more refined and higher in quality compared to the ones generated locally, even though both are using the same model weights.

I was wondering if there might be any optimizations, fine-tuning, or special adjustments made to the hosted version that could be affecting the output. Could the demo be using additional techniques or post-processing to enhance the results?

Any insight into this would be really helpful. Thanks in advance!

edit: I am using vLLM for inference, I do not know if this makes a difference, also tried passing the transformers implementation flag to vLLM but is not supported for molmo