albermax / albermax/innvestigate

Deep Taylor default setting violates conservation property

未關閉
#161 0 則留言 2 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
1.3k
分支
230
PR 合併指標
30 天內沒有已合併 PR

描述

Dear team,
i finally got some time to look into your latest version of iNNvestigate. Thank you very much for providing the package!

I want to use DTD and I wonder how the [current implementation](https://github.com/albermax/innvestigate/blob/1ed38a377262236981090bb0989d2e1a6892a0b1/innvestigate/analyzer/deeptaylor.py#L31) currently works exactly:

- Do you choose according to the layer-type or activation type different root points corresponding to decomposition rules?
- I see that you pick lrp-rules, but I do not really get the logic of your implementation.

- In your old implementation you wanted to change the basic DTD setup, see [here](https://github.com/albermax/innvestigate/issues/91#issuecomment-414376522). Might it be the changed got lost?

Based on your [introduction notebook](https://github.com/albermax/innvestigate/blob/master/examples/notebooks/introduction.ipynb) I started to check the conversation property. Full code including your utiliy functions is here.

-i Test image is a 9

### Having no bias constraint
- some relevances is "leaking" as described by [Sebastian Lapuschkin](https://github.com/albermax/innvestigate/issues/91#issuecomment-414376522).
- [Source](https://github.com/enryH/testing_iNNvestigate/blob/master/testing_dtd_wo_bias_constraint.py)

```python
# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:.3f}".format(analysis.sum()))
try:
assert abs(scores.max() - analysis.sum()) < 0.001
except AssertionError:
print("not equal...")
# Biases are included and conversation property in DTD framework fails
```
```
Maximum Score: 11.711 with label 9
sum of relevances assigned to inputs: 11.561
not equal...
```

```python
# LRP-Alpha_1-Beta_0 without biases is z+ rule in DTD paper
from innvestigate.analyzer.relevance_based.relevance_analyzer import LRPAlpha1Beta0IgnoreBias
analyzer = LRPAlpha1Beta0IgnoreBias(model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:3f}".format(analysis.sum()))
assert abs(scores.max() - analysis.sum()) < 0.001
```

```
Maximum Score: 11.711 with label 9
sum of relevances assigned to inputs: 11.711
```

### Constrainig the bias in ReLUs to be negative as in DTD paper
- relevance over inputs is larger as output score using default setup
- [Source](https://github.com/enryH/testing_iNNvestigate/blob/master/testing_dtd_w_bias_constraint.py)
```python
# Create analyzer
analyzer = innvestigate.create_analyzer("deep_taylor", model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:.3f}".format(analysis.sum()))
try:
assert abs(scores.max() - analysis.sum()) < 0.001
except AssertionError:
print("not equal...")
# Biases are included and conversation property in DTD framework fails
```
```
Maximum Score: 12.835 with label 9
sum of relevances assigned to inputs: 13.338
not equal...
```
```python
# LRP-Alpha_1-Beta_0 without biases is z+ rule in DTD paper
from innvestigate.analyzer.relevance_based.relevance_analyzer import LRPAlpha1Beta0IgnoreBias
analyzer = LRPAlpha1Beta0IgnoreBias(model_wo_sm)

# Applying the analyzer
analysis = analyzer.analyze(image)

# Check Conservation
scores = model_wo_sm.predict(image)
print("Maximum Score: {:.3f} with label {}".format(scores.max(), scores.argmax()))
print("sum of relevances assigned to inputs: {:3f}".format(analysis.sum()))
assert abs(scores.max() - analysis.sum()) < 0.001
```

```
Maximum Score: 12.835 with label 9
sum of relevances assigned to inputs: 12.835
```

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。