albermax / albermax/innvestigate

Deep Taylor default setting violates conservation property

Abierto
#161 0 comentarios 2 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
1.3k
Forks
230
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Dear team,
i finally got some time to look into your latest version of iNNvestigate. Thank you very much for providing the package!

I want to use DTD and I wonder how the [current implementation](https://github.com/albermax/innvestigate/blob/1ed38a377262236981090bb0989d2e1a6892a0b1/innvestigate/analyzer/deeptaylor.py#L31) currently works exactly:

- Do you choose according to the layer-type or activation type different root points corresponding to decomposition rules?
- I see that you pick lrp-rules, but I do not really get the logic of your implementation.

- In your old implementation you wanted to change the basic DTD setup, see [here](https://github.com/albermax/innvestigate/issues/91#issuecomment-414376522). Might it be the changed got lost?

Based on your [introduction notebook](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/introduction.ipynb) I started to check the conversation property. Full code including your utiliy functions is here.

-i Test image is a 9

### Having no bias constraint
- some relevances is "leaking" as described by [Sebastian Lapuschkin](https://github.com/albermax/innvestigate/issues/91#issuecomment-414376522).
- [Source](https://github.com/enryH/testing_iNNvestigate/blob/master/testing_dtd_wo_bias_constraint.py)

```python
# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:.3f}".format(analysis.sum()))
try:
assert abs(scores.max() - analysis.sum()) < 0.001
except AssertionError:
print("not equal...")
# Biases are included and conversation property in DTD framework fails
```
```
Maximum Score: 11.711 with label 9
sum of relevances assigned to inputs: 11.561
not equal...
```

```python
# LRP-Alpha_1-Beta_0 without biases is z+ rule in DTD paper
from innvestigate.analyzer.relevance_based.relevance_analyzer import LRPAlpha1Beta0IgnoreBias
analyzer = LRPAlpha1Beta0IgnoreBias(model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:3f}".format(analysis.sum()))
assert abs(scores.max() - analysis.sum()) < 0.001
```

```
Maximum Score: 11.711 with label 9
sum of relevances assigned to inputs: 11.711
```

### Constrainig the bias in ReLUs to be negative as in DTD paper
- relevance over inputs is larger as output score using default setup
- [Source](https://github.com/enryH/testing_iNNvestigate/blob/master/testing_dtd_w_bias_constraint.py)
```python
# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:.3f}".format(analysis.sum()))
try:
assert abs(scores.max() - analysis.sum()) < 0.001
except AssertionError:
print("not equal...")
# Biases are included and conversation property in DTD framework fails
```
```
Maximum Score: 12.835 with label 9
sum of relevances assigned to inputs: 13.338
not equal...
```
```python
# LRP-Alpha_1-Beta_0 without biases is z+ rule in DTD paper
from innvestigate.analyzer.relevance_based.relevance_analyzer import LRPAlpha1Beta0IgnoreBias
analyzer = LRPAlpha1Beta0IgnoreBias(model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:3f}".format(analysis.sum()))
assert abs(scores.max() - analysis.sum()) < 0.001
```

```
Maximum Score: 12.835 with label 9
sum of relevances assigned to inputs: 12.835
```

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.