aai-institute / aai-institute/pyDVL
Flatten for eigen methods
- Ngôn ngữ chính
- Python
- Star
- 146
- Fork
- 10
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
When using eigsh routines from scipy or cupy, we have to output a flat tensor from the matvec function. However, when calling the hvp on the functional call to the model, the output is a dict of tensors. Right now, we have to flatten after each hvp call, e.g. for one batch:
```python
def batch_hvp(vec: torch.Tensor):
return flatten_tensors_to_vector(
hvp(
batch_loss,
model_params,
align_structure(model_params, vec),
reverse_only=reverse_only,
).values()
)
```
We should investigate the runtime and memory consequences of this and if there is an alternative for it.
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.