albermax / albermax/innvestigate
Deep Taylor default setting violates conservation property
- Lenguaje dominante
- Python
- Estrellas
- 1.3k
- Forks
- 230
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Dear team,
i finally got some time to look into your latest version of iNNvestigate. Thank you very much for providing the package!
I want to use DTD and I wonder how the [current implementation](https://github.com/albermax/innvestigate/blob/1ed38a377262236981090bb0989d2e1a6892a0b1/innvestigate/analyzer/deeptaylor.py#L31) currently works exactly:
- Do you choose according to the layer-type or activation type different root points corresponding to decomposition rules?
- I see that you pick lrp-rules, but I do not really get the logic of your implementation.
- In your old implementation you wanted to change the basic DTD setup, see [here](https://github.com/albermax/innvestigate/issues/91#issuecomment-414376522). Might it be the changed got lost?
Based on your [introduction notebook](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/introduction.ipynb) I started to check the conversation property. Full code including your utiliy functions is here.
-i Test image is a 9
### Having no bias constraint
- some relevances is "leaking" as described by [Sebastian Lapuschkin](https://github.com/albermax/innvestigate/issues/91#issuecomment-414376522).
- [Source](https://github.com/enryH/testing_iNNvestigate/blob/master/testing_dtd_wo_bias_constraint.py)
```python
# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model_wo_sm)
# Applying the analyzer
analysis = analyzer.analyze(image)
# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:.3f}".format(analysis.sum()))
try:
assert abs(scores.max() - analysis.sum()) < 0.001
except AssertionError:
print("not equal...")
# Biases are included and conversation property in DTD framework fails
```
```
Maximum Score: 11.711 with label 9
sum of relevances assigned to inputs: 11.561
not equal...
```
```python
# LRP-Alpha_1-Beta_0 without biases is z+ rule in DTD paper
from innvestigate.analyzer.relevance_based.relevance_analyzer import LRPAlpha1Beta0IgnoreBias
analyzer = LRPAlpha1Beta0IgnoreBias(model_wo_sm)
# Applying the analyzer
analysis = analyzer.analyze(image)
# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:3f}".format(analysis.sum()))
assert abs(scores.max() - analysis.sum()) < 0.001
```
```
Maximum Score: 11.711 with label 9
sum of relevances assigned to inputs: 11.711
```
### Constrainig the bias in ReLUs to be negative as in DTD paper
- relevance over inputs is larger as output score using default setup
- [Source](https://github.com/enryH/testing_iNNvestigate/blob/master/testing_dtd_w_bias_constraint.py)
```python
# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model_wo_sm)
# Applying the analyzer
analysis = analyzer.analyze(image)
# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:.3f}".format(analysis.sum()))
try:
assert abs(scores.max() - analysis.sum()) < 0.001
except AssertionError:
print("not equal...")
# Biases are included and conversation property in DTD framework fails
```
```
Maximum Score: 12.835 with label 9
sum of relevances assigned to inputs: 13.338
not equal...
```
```python
# LRP-Alpha_1-Beta_0 without biases is z+ rule in DTD paper
from innvestigate.analyzer.relevance_based.relevance_analyzer import LRPAlpha1Beta0IgnoreBias
analyzer = LRPAlpha1Beta0IgnoreBias(model_wo_sm)
# Applying the analyzer
analysis = analyzer.analyze(image)
# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:3f}".format(analysis.sum()))
assert abs(scores.max() - analysis.sum()) < 0.001
```
```
Maximum Score: 12.835 with label 9
sum of relevances assigned to inputs: 12.835
```
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.