Gens are inconsistent on complex prompts

Seems to only happen on complex prompt with lots of varying weights. It doesn't happen on [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui) btw.

## Reproduction
1. Generate image 1
2. Add a comma or something to the prompt to modify it -> Queue Prompt -> then quickly cancel.
3. Undo your previous modification to the prompt.
4. Try to generate image 1 again.

## Expectation
The same image.

## What I got
A slightly different image.