albermax / albermax/innvestigate

Smoothgrad outputs only zeros for a batch of inputs

Offen
#246 0 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @adrhill Auf GitHub ansehen
bug
Vorherrschende Sprache
Python
Sterne
1.3k
Forks
230
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Hello!

I tried to analyze a batch of inputs with the 'smoothgrad' method and `neuron_selection_mode = "index"` with your package iNNvestigate (version 1.0.9). Surprisingly, only the output for the first input in the batch made any sense, because the gradients for all other values were set to 0. I think there's something wrong with your code.

Here is a small example:
I generated a classification dataset with `sklearn` with 5 input features and 2 classes. On this dataset, I trained a keras model with 3 dense layers. Then I removed the `softmax` activation and created the analyzers for the methods 'gradient' and 'smoothgrad'. I also set the noise scale (`noise_scale = 1e-8`) to a very small value for the `smoothgrad` method, which makes the results almost identical to the normally calculated gradients. Below you can find the code for my example:

```python
import numpy as np
np.random.seed(1234)

from sklearn.datasets import make_classification

import keras # version 2.2.4
import innvestigate # version 1.0.9
import innvestigate.utils as iutils

n_inputs = 5
n_classes = 2
n_samples = 512

# Generate classification data
data_x, data_y = make_classification(n_samples=n_samples, n_features=n_inputs, n_classes=n_classes)
data_y = keras.utils.to_categorical(data_y)

# Define keras model and fit it
model = keras.models.Sequential([
keras.layers.Dense(16, activation="relu", input_shape=(n_inputs,)),
keras.layers.Dense(8, activation="relu"),
keras.layers.Dense(n_classes, activation="softmax"),
])
model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])
model.fit(data_x, data_y, epochs=25, batch_size=128)

# Start the analysis with iNNvestigate
model = iutils.keras.graph.model_wo_softmax(model)

# Set the noise level sufficiently low such that almost identical values should come out for the methods
# 'gradient' and 'smoothgrad'
noise_scale = 1e-8

# Define the analyzer for each method
analyzer_smoothgrad = innvestigate.create_analyzer("smoothgrad",
model,
noise_scale = noise_scale,
neuron_selection_mode="index")
analyzer_gradient = innvestigate.create_analyzer("gradient",
model,
neuron_selection_mode="index")

# One input for output neuron '0'
result_smoothgrad = analyzer_smoothgrad.analyze(data_x[1:2,:], 0)
result_gradient = analyzer_gradient.analyze(data_x[1:2,:], 0)
print("Mean squared error (one input): {:4f}".format(np.mean((result_gradient - result_smoothgrad)**2)))

# Multiple inputs for output neuron '0'
result_smoothgrad = analyzer_smoothgrad.analyze(data_x[1:3,:], 0)
result_gradient = analyzer_gradient.analyze(data_x[1:3,:], 0)
print("Mean squared error (multiple inputs): {:4f}".format(np.mean((result_gradient - result_smoothgrad)**2)))

print("Result smoothgrad:")
print(result_smoothgrad)

print("Result gradient:")
print(result_gradient)
```
Here is the output from the example and as you can see the output of 'smoothgrad' for the second input makes no sense:
```bash
Mean squared error (one input): 0.000000
Mean squared error (multiple inputs): 0.225435
Result smoothgrad:
[[ 0.50317013 0.72440994 0.59556705 -1.5030893 -0.39739025]
[ 0. 0. 0. 0. 0. ]]
Result gradient:
[[ 0.5031703 0.72440976 0.5955669 -1.5030882 -0.39739007]
[ 0.46457082 -0.31459585 0.5895394 -1.172667 -0.46567515]]
```
I hope I didn't make a mistake and that I could help you to improve this wonderful package.

Best regards,
Niklas

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.