Question for DDIM sampling configuration
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 45/100
- Tipo de issue
- Error
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Stack tecnológico
- python
- Área
- machine-learning
Línea de trabajo
Compara configs/model/base.yaml y configs/rawdata_diffusion_sampling.yaml, y luego sigue resolve_sampling_runner hasta src/models/diffusion/diffusion_sampling.py. Reproduce la secuencia de pasos de tiempo configurada e inspecciona cómo se usa alphas_cumprod_prev. Se considerará terminado cuando se confirme la semántica prevista del tiempo de inicio o se especifique, implemente y cubra con una prueba adecuada un schedule respaced.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Hi, thank you for releasing the PerturbDiff implementation. I noticed a possible mismatch between the DDIM sampling configuration and the training diffusion schedule.
The model is configured with a 1000-step diffusion process:
[configs/model/base.yaml](https://github.com/DeepGraphLearning/PerturbDiff/blob/main/configs/model/base.yaml#L34-L38):steps: 1000
However, the default sampling configuration uses:
[configs/rawdata_diffusion_sampling.yaml](https://github.com/DeepGraphLearning/PerturbDiff/blob/main/configs/rawdata_diffusion_sampling.yaml#L38-L42):start_time: 100
In [resolve_sampling_runner](https://github.com/DeepGraphLearning/PerturbDiff/blob/main/src/apps/sampling/sampling_generation_helpers.py#L23-L34), this value is passed directly to ddim_sample_loop as start_time. The DDIM loop then:
- Initializes the state from standard Gaussian noise:
img = noise if noise is not None else th.randn(*shape, device=device)
- Constructs consecutive timestep indices:
indices = list(range(start_time))[::-1]
Therefore, with the default configuration, sampling starts from pure Gaussian noise at timestep 99 and performs 100 consecutive updates:
99 -> 98 -> ... -> 1 -> 0
This behavior is implemented in [ddim_sample_loop_progressive](https://github.com/DeepGraphLearning/PerturbDiff/blob/main/src/models/diffusion/diffusion_sampling.py#L559-L590). Each DDIM update also uses alphas_cumprod_prev[t], so it specifically transitions from timestep t to the adjacent timestep t-1, rather than between respaced timesteps:
My concern is that, under the default 1000-step linear schedule, timestep 99 is not close to the terminal Gaussian distribution. The configured beta schedule gives approximately:
alpha_bar[99] = 0.897
sqrt(alpha_bar[99]) = 0.947
sqrt(1 - alpha_bar[99]) = 0.321
Therefore, the forward-process state at timestep 99 is approximately:
x_99 = 0.947 * x_0 + 0.321 * noise
In other words, the training distribution at timestep 99 still contains a strong contribution from the clean sample. The sampler instead initializes:
x_99 ~ Normal(0, I)
If the intended goal is accelerated DDIM sampling with 100 model evaluations, I would expect the sampler to select approximately 100 respaced timesteps spanning the full training horizon from 999 to 0, and to calculate each update using the previous selected timestep. The current implementation instead appears to treat start_time as both the desired number of sampling steps and the actual starting diffusion timestep.
Could you please explain the rationale for initializing pure Gaussian noise at timestep 99? Is this behavior intentional, or should the 100-step DDIM sampler use a respaced timestep sequence covering the full 1000-step training schedule?
Thank you for your time and clarification.
- Lenguaje dominante
- Python
- Estrellas
- 63
- Forks
- 10
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de DeepGraphLearning/PerturbDiff
-
Dificultad 4/5 3-5 días Aptitud para principiantes 38/100
DeepGraphLearning/PerturbDiff#7 · 1 comentario ·
-
Question: Simulating gene knockout on novel datasets (zero-shot inference) & Implementation feedback Abierto
Dificultad 5/5 Más de una semana Aptitud para principiantes 30/100
DeepGraphLearning/PerturbDiff#5 · 1 comentario · 1 reacción ·
-
RuntimeError: mat1 and mat2 shapes cannot be multiplied during sampling with finetuned_replogle.ckpt Abierto
Dificultad 3/5 1-2 días Aptitud para principiantes 68/100
DeepGraphLearning/PerturbDiff#4 · 3 comentarios ·
-
Inference Abierto
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
DeepGraphLearning/PerturbDiff#1 · 4 comentarios ·
Todos los issues de DeepGraphLearning/PerturbDiff
Issues similares
-
link-check link-check:sphinx-theme
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
qgis/QGIS-Documentation#11275 ·
-
bug priority:normal ready-for-dev
Dificultad 2/5 1-3 horas Aptitud para principiantes 88/100
OpenHands/extensions#626 · 1 comentario ·
-
Change observation tooltip text Abierto
Dificultad 1/5 Menos de una hora Aptitud para principiantes 90/100
CSCfi/sd-search-api#39 ·
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 90/100