Upscale time still very slow

I have reported this issue a months ago. and during the time I haven't use it at all cuz the first time I reported it. it still haven't fixed for weeks so I just quit and think that I'll come back later

same issue as https://github.com/comfyanonymous/ComfyUI/issues/858 and https://github.com/comfyanonymous/ComfyUI/issues/1294
when generating the first pic it takes only 40-50s but when It sends the pic to the upscale (the second Ksampler) the time used is usually 7-8 s/it but now it is 50-70 s/it
Fix this pls
it is unuseable T_T

the sampling method is dpmpp_2m_sde-karras on all the sampler
preview method Taesd

this is the image


can you fix some code like what you did on https://github.com/comfyanonymous/ComfyUI/issues/858#issuecomment-1627726371 ?
this is the prompt window, the one that have 1:30 upscale time is because I set the first sampler (first generate to dpmpp_2m I think, that makes the time goes up to 7 minute and the upscale time went back to normal) but the face detailer still use a lot of time as the same...

edit** the Sampler are irrelevant. I just did one that are 3 minute at first generation, and normal speed on upscale and face detailer, but then I redo the same setting, prompt, same seed and it got slower