Remove redundant comparison inside the diffusion loop of stable video diffusion pipeline

**Is your feature request related to a problem? Please describe.**
I found that the inside the `__call__` of stable video diffusion keeps doing async memcpy between host to device as attached.

**Describe the solution you'd like.**
The reason for that is actually coming from every time we get `self.do_classifier_free_guidance`, we compared tensor between `int` -> get boolean on device -> memcpy that boolean from gpu to cpu.

It'll be good to just assign a variable for it before the loop as the value won't change through the loop.

**Additional context.**
I'm glad to contribute this by opening a PR

Start at the stable video diffusion pipeline's __call__ method and inspect the diffusion loop where self.do_classifier_free_guidance is repeatedly evaluated. Cache the unchanged value before the loop, then verify the loop no longer performs that repeated comparison or associated host-device transfer.

Vom Indexierungsmodell aus dem Issue-Text verfasst.