gaozhihan / gaozhihan/PreDiff

Question about the design of shifted predicted mean

Open
#16 1 comment 0 reactions 1 assignee Claimed by @gaozhihan View on GitHub
question
Dominant language
Python
Stars
159
Forks
10
PR merge metrics
No merged PRs in 30d

Description

I have noticed that after training the knowledge control network, in the paper, you mentioned that we need to shift the predicted mean by $-\lambda_{\mathcal{F}}\sum_{\theta}\\nabla_{z_t}||U_{\phi}(z_t, t, y) - \mathcal{F}\_0(y)||$. However, I think that the term $||U_{\phi}(z_t, t, y) - \mathcal{F}\_0(y)||$ should be closed to zero because this is the exact training objective of the knowledge control network. Therefore, the gradient would be very close to zero.

May I ask why you would design the form like this?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.