albermax / albermax/innvestigate

Deep Taylor default setting violates conservation property

Aberta
#161 0 comentários 2 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
1.3k
Forks
230
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

Dear team,
i finally got some time to look into your latest version of iNNvestigate. Thank you very much for providing the package!

I want to use DTD and I wonder how the [current implementation](https://github.com/albermax/innvestigate/blob/1ed38a377262236981090bb0989d2e1a6892a0b1/innvestigate/analyzer/deeptaylor.py#L31) currently works exactly:

- Do you choose according to the layer-type or activation type different root points corresponding to decomposition rules?
- I see that you pick lrp-rules, but I do not really get the logic of your implementation.

- In your old implementation you wanted to change the basic DTD setup, see [here](https://github.com/albermax/innvestigate/issues/91#issuecomment-414376522). Might it be the changed got lost?

Based on your [introduction notebook](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/introduction.ipynb) I started to check the conversation property. Full code including your utiliy functions is here.

-i Test image is a 9

### Having no bias constraint
- some relevances is "leaking" as described by [Sebastian Lapuschkin](https://github.com/albermax/innvestigate/issues/91#issuecomment-414376522).
- [Source](https://github.com/enryH/testing_iNNvestigate/blob/master/testing_dtd_wo_bias_constraint.py)

```python
# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:.3f}".format(analysis.sum()))
try:
assert abs(scores.max() - analysis.sum()) < 0.001
except AssertionError:
print("not equal...")
# Biases are included and conversation property in DTD framework fails
```
```
Maximum Score: 11.711 with label 9
sum of relevances assigned to inputs: 11.561
not equal...
```

```python
# LRP-Alpha_1-Beta_0 without biases is z+ rule in DTD paper
from innvestigate.analyzer.relevance_based.relevance_analyzer import LRPAlpha1Beta0IgnoreBias
analyzer = LRPAlpha1Beta0IgnoreBias(model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:3f}".format(analysis.sum()))
assert abs(scores.max() - analysis.sum()) < 0.001
```

```
Maximum Score: 11.711 with label 9
sum of relevances assigned to inputs: 11.711
```

### Constrainig the bias in ReLUs to be negative as in DTD paper
- relevance over inputs is larger as output score using default setup
- [Source](https://github.com/enryH/testing_iNNvestigate/blob/master/testing_dtd_w_bias_constraint.py)
```python
# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:.3f}".format(analysis.sum()))
try:
assert abs(scores.max() - analysis.sum()) < 0.001
except AssertionError:
print("not equal...")
# Biases are included and conversation property in DTD framework fails
```
```
Maximum Score: 12.835 with label 9
sum of relevances assigned to inputs: 13.338
not equal...
```
```python
# LRP-Alpha_1-Beta_0 without biases is z+ rule in DTD paper
from innvestigate.analyzer.relevance_based.relevance_analyzer import LRPAlpha1Beta0IgnoreBias
analyzer = LRPAlpha1Beta0IgnoreBias(model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:3f}".format(analysis.sum()))
assert abs(scores.max() - analysis.sum()) < 0.001
```

```
Maximum Score: 12.835 with label 9
sum of relevances assigned to inputs: 12.835
```

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.