albermax / albermax/innvestigate

Error in defining analyzer for Leaky Relu (Advanced activation Layers)

Aberta
#138 4 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
1.3k
Forks
230
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

Hey Alber,

I am facing some error in defining analyzers whenever there is LeakyRelu activation in my model definition (I think this error occurs whenever there is some advanced activation layer but I am not sure).

Here is the code to reproduce the problem.

```
from keras.utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense, Activation, Flatten
from keras.layers import Conv2D, LeakyReLU
import keras
from keras import backend as K

import innvestigate
import innvestigate.utils as iutils

# Model With Relu Activation
########################################
K.clear_session()
model1 = Sequential()
model1.add( Conv2D( filters=64, kernel_size=(3, 3), strides=(1,1), input_shape=(3, 3, 1) ) )
model1.add(Activation('relu'))
model1.add(Flatten())
model1.add(Dense(10))
model1.add(Activation('softmax'))
print('Here is the summary of model 1 with ReLu Activation')
model1.summary()

#Heatmap for model1
############################
methods = ['gradient']
model1_wo_softmax = iutils.keras.graph.model_wo_softmax(model1)

print('Defining Alalyzer for model 1')
analyzer_model1 = innvestigate.create_analyzer(methods[0], model1_wo_softmax)
print('Success\n')

###############################################################################################

# Model with LeakyRelu Activation
#########################################
model2 = Sequential()
model2.add( Conv2D( filters=64, kernel_size=(3, 3), strides=(1,1), input_shape=(3, 3, 1) ) )
model2.add(LeakyReLU(alpha=0.01))
model2.add(Flatten())
model2.add(Dense(10))
model2.add(Activation('softmax'))
print('Here is the summary of model 2 with leaky ReLu Activation')
model2.summary()

#Heatmap for model2
############################
methods = ['gradient']
model2_wo_softmax = iutils.keras.graph.model_wo_softmax(model2)

print('Defining Alalyzer for model 2')
analyzer_model2 = innvestigate.create_analyzer(methods[0], model2_wo_softmax)
print('Success')
```

Here is the results/error that I am getting.
```
Using TensorFlow backend.
Here is the summary of model 1 with ReLu Activation
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_1 (Conv2D) (None, 1, 1, 64) 640
_________________________________________________________________
activation_1 (Activation) (None, 1, 1, 64) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 64) 0
_________________________________________________________________
dense_1 (Dense) (None, 10) 650
_________________________________________________________________
activation_2 (Activation) (None, 10) 0
=================================================================
Total params: 1,290
Trainable params: 1,290
Non-trainable params: 0
_________________________________________________________________
Defining Alalyzer for model 1
Success

Here is the summary of model 2 with leaky ReLu Activation
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_2 (Conv2D) (None, 1, 1, 64) 640
_________________________________________________________________
leaky_re_lu_1 (LeakyReLU) (None, 1, 1, 64) 0
_________________________________________________________________
flatten_2 (Flatten) (None, 64) 0
_________________________________________________________________
dense_2 (Dense) (None, 10) 650
_________________________________________________________________
activation_3 (Activation) (None, 10) 0
=================================================================
Total params: 1,290
Trainable params: 1,290
Non-trainable params: 0
_________________________________________________________________
Defining Alalyzer for model 2
Traceback (most recent call last):
File "innvestigate_issue.py", line 51, in
analyzer_model2 = innvestigate.create_analyzer(methods[0], model2_wo_softmax)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/__init__.py", line 99, in create_analyzer
return analyzers[name](model, **kwargs)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/gradient_based.py", line 119, in __init__
super(Gradient, self).__init__(model, **kwargs)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 597, in __init__
super(ReverseAnalyzerBase, self).__init__(model, **kwargs)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 347, in __init__
super(AnalyzerNetworkBase, self).__init__(model, **kwargs)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 79, in __init__
self._do_model_checks()
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 104, in _do_model_checks
checked = kgraph.model_contains(self._model, check)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/utils/keras/graph.py", line 344, in model_contains
tmp = [layer for layer in layers if condition(layer)]
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/utils/keras/graph.py", line 344, in
tmp = [layer for layer in layers if condition(layer)]
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 355, in
layer, activation="softmax"),
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/utils/keras/checks.py", line 226, in contains_activation
raise Exception("Cannot detect activation type.")
Exception: Cannot detect activation type.
```

It is somehow confusing the LeakyRelu with Softmax activation and thus, fails to apply `model_wo_softmax` function on it.

Could you please help me fix this issue. I am stuck with this part since my model uses LeakyRelu activations. I would be grateful to you.

Thanks,
Naman

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.