Question about Hessian Matrix Calculation
- Dominant language
- Python
- Stars
- 158
- Forks
- 39
- PR merge metrics
- No merged PRs in 30d
Description
Hello, really appreciate your nice work.
I hope this message finds you well. I have two questions regarding the calculation of the Hessian matrix in your code. Specifically, I'm looking at the [function](https://github.com/automl/RobustDARTS/blob/3dec3fedaa1770614bbfd4b98f5299d946ea7ac6/src/search/analyze.py#L157) where you calculate the second-order derivatives for each parameter with respect to all parameters:
```Python
row = self.gradient(grad[j], inputs[i:], retain_graph=True)[j:]
```
(1) I wonder why only the [j:] part of the result is taken? Is it assumed that the derivative has no effect on the preceding parameters?
(2) Additionally, when assigning values, why is the assignment done as follows and could you please explain the reasoning behind these specific assignments?
```Python
out.data[ai, ai:].add_(row.clone().type_as(out).data) # ai's row
if ai + 1 < n:
out.data[ai + 1:, ai].add_(row.clone().type_as(out).data[1:]) # ai's column
```
Thank you very much for your time and effort in maintaining this project. Your help is greatly appreciated.
Best regards,
Shun Lu
Contributor guide
No contributing guide indexed for this repository
Research direction
Read src/search/analyze.py at the linked line and trace the gradient calls that produce row and out. Determine how the j: slice and the row/column assignments represent the Hessian calculation, then document the reasoning in the issue or nearby documentation. No test or other entry point is mentioned.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100