anthropics / anthropics/jacobian-lens

Interest in a contributed swap-intervention implementation?

Abierto
#2 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
1.9k
Forks
282
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

I'm replicating the workspace paper with this repo (the shipped prompt sets and walkthrough made that much easier, thanks). One gap: `data/experiments/README.md` specifies the swap intervention precisely, and probe-swap, flexible-generalization and top-down-summoning all depend on it, but the package only ships the read side (`ActivationRecorder` / `JacobianLens.apply`). Anyone reproducing the causal results has to re-implement the intervention hooks themselves.

I have a tested implementation from my replication: forward hooks that clamp/replace a lens coordinate across a layer band (direction `normalize(J_l^T W_U[t])` with coefficient transfer, applied at prompt positions), with the direction source parameterized so linear-probe directions can be swapped in for the probe-swap protocol. I'd be happy to adapt it into a PR, either as a `jlens` module or as an `examples/` script, whichever fits your intent for this repo.

Is that something you'd take? If interventions are deliberately out of scope for the package, that's useful to know too, and feel free to close.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.