aai-institute / aai-institute/pyDVL

Flatten for eigen methods

未关闭
#401 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
benchmarking enhancement
主要语言
Python
星标
146
派生
10
PR 合并指标
30 天内没有已合并 PR

描述

When using eigsh routines from scipy or cupy, we have to output a flat tensor from the matvec function. However, when calling the hvp on the functional call to the model, the output is a dict of tensors. Right now, we have to flatten after each hvp call, e.g. for one batch:

```python
def batch_hvp(vec: torch.Tensor):
return flatten_tensors_to_vector(
hvp(
batch_loss,
model_params,
align_structure(model_params, vec),
reverse_only=reverse_only,
).values()
)
```

We should investigate the runtime and memory consequences of this and if there is an alternative for it.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。