anthropics / anthropics/jacobian-lens

Interest in a contributed swap-intervention implementation?

Ouverte
#2 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
1.9k
Forks
282
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

I'm replicating the workspace paper with this repo (the shipped prompt sets and walkthrough made that much easier, thanks). One gap: `data/experiments/README.md` specifies the swap intervention precisely, and probe-swap, flexible-generalization and top-down-summoning all depend on it, but the package only ships the read side (`ActivationRecorder` / `JacobianLens.apply`). Anyone reproducing the causal results has to re-implement the intervention hooks themselves.

I have a tested implementation from my replication: forward hooks that clamp/replace a lens coordinate across a layer band (direction `normalize(J_l^T W_U[t])` with coefficient transfer, applied at prompt positions), with the direction source parameterized so linear-probe directions can be swapped in for the probe-swap protocol. I'd be happy to adapt it into a PR, either as a `jlens` module or as an `examples/` script, whichever fits your intent for this repo.

Is that something you'd take? If interventions are deliberately out of scope for the package, that's useful to know too, and feel free to close.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Piste de recherche

Start with data/experiments/README.md and the shipped ActivationRecorder and JacobianLens.apply entry points; compare the documented protocols with the replication's tested forward-hook implementation. Confirm with maintainers whether interventions belong in jlens or examples/, then define acceptance around reproducing probe-swap, flexible-generalization, and top-down-summoning.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python
Domaine
machine-learning
Type d'issue
Fonctionnalité
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
Calme
Clarté
À clarifier
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.