aai-institute / aai-institute/pyDVL

Flatten for eigen methods

Abierto
#401 0 comentarios 0 reacciones 0 asignados Ver en GitHub
benchmarking enhancement
Lenguaje dominante
Python
Estrellas
146
Forks
10
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

When using eigsh routines from scipy or cupy, we have to output a flat tensor from the matvec function. However, when calling the hvp on the functional call to the model, the output is a dict of tensors. Right now, we have to flatten after each hvp call, e.g. for one batch:

```python
def batch_hvp(vec: torch.Tensor):
return flatten_tensors_to_vector(
hvp(
batch_loss,
model_params,
align_structure(model_params, vec),
reverse_only=reverse_only,
).values()
)
```

We should investigate the runtime and memory consequences of this and if there is an alternative for it.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.