albermax / albermax/innvestigate
Error in defining analyzer for Leaky Relu (Advanced activation Layers)
- Linguagem predominante
- Python
- Estrelas
- 1.3k
- Forks
- 230
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
Hey Alber,
I am facing some error in defining analyzers whenever there is LeakyRelu activation in my model definition (I think this error occurs whenever there is some advanced activation layer but I am not sure).
Here is the code to reproduce the problem.
```
from keras.utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense, Activation, Flatten
from keras.layers import Conv2D, LeakyReLU
import keras
from keras import backend as K
import innvestigate
import innvestigate.utils as iutils
# Model With Relu Activation
########################################
K.clear_session()
model1 = Sequential()
model1.add( Conv2D( filters=64, kernel_size=(3, 3), strides=(1,1), input_shape=(3, 3, 1) ) )
model1.add(Activation('relu'))
model1.add(Flatten())
model1.add(Dense(10))
model1.add(Activation('softmax'))
print('Here is the summary of model 1 with ReLu Activation')
model1.summary()
#Heatmap for model1
############################
methods = ['gradient']
model1_wo_softmax = iutils.keras.graph.model_wo_softmax(model1)
print('Defining Alalyzer for model 1')
analyzer_model1 = innvestigate.create_analyzer(methods[0], model1_wo_softmax)
print('Success\n')
###############################################################################################
# Model with LeakyRelu Activation
#########################################
model2 = Sequential()
model2.add( Conv2D( filters=64, kernel_size=(3, 3), strides=(1,1), input_shape=(3, 3, 1) ) )
model2.add(LeakyReLU(alpha=0.01))
model2.add(Flatten())
model2.add(Dense(10))
model2.add(Activation('softmax'))
print('Here is the summary of model 2 with leaky ReLu Activation')
model2.summary()
#Heatmap for model2
############################
methods = ['gradient']
model2_wo_softmax = iutils.keras.graph.model_wo_softmax(model2)
print('Defining Alalyzer for model 2')
analyzer_model2 = innvestigate.create_analyzer(methods[0], model2_wo_softmax)
print('Success')
```
Here is the results/error that I am getting.
```
Using TensorFlow backend.
Here is the summary of model 1 with ReLu Activation
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_1 (Conv2D) (None, 1, 1, 64) 640
_________________________________________________________________
activation_1 (Activation) (None, 1, 1, 64) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 64) 0
_________________________________________________________________
dense_1 (Dense) (None, 10) 650
_________________________________________________________________
activation_2 (Activation) (None, 10) 0
=================================================================
Total params: 1,290
Trainable params: 1,290
Non-trainable params: 0
_________________________________________________________________
Defining Alalyzer for model 1
Success
Here is the summary of model 2 with leaky ReLu Activation
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_2 (Conv2D) (None, 1, 1, 64) 640
_________________________________________________________________
leaky_re_lu_1 (LeakyReLU) (None, 1, 1, 64) 0
_________________________________________________________________
flatten_2 (Flatten) (None, 64) 0
_________________________________________________________________
dense_2 (Dense) (None, 10) 650
_________________________________________________________________
activation_3 (Activation) (None, 10) 0
=================================================================
Total params: 1,290
Trainable params: 1,290
Non-trainable params: 0
_________________________________________________________________
Defining Alalyzer for model 2
Traceback (most recent call last):
File "innvestigate_issue.py", line 51, in
analyzer_model2 = innvestigate.create_analyzer(methods[0], model2_wo_softmax)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/__init__.py", line 99, in create_analyzer
return analyzers[name](model, **kwargs)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/gradient_based.py", line 119, in __init__
super(Gradient, self).__init__(model, **kwargs)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 597, in __init__
super(ReverseAnalyzerBase, self).__init__(model, **kwargs)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 347, in __init__
super(AnalyzerNetworkBase, self).__init__(model, **kwargs)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 79, in __init__
self._do_model_checks()
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 104, in _do_model_checks
checked = kgraph.model_contains(self._model, check)
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/utils/keras/graph.py", line 344, in model_contains
tmp = [layer for layer in layers if condition(layer)]
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/utils/keras/graph.py", line 344, in
tmp = [layer for layer in layers if condition(layer)]
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/analyzer/base.py", line 355, in
layer, activation="softmax"),
File "/home/cse_h1/nzb0040/miniconda3/envs/innvestigate_latest/lib/python3.6/site-packages/innvestigate/utils/keras/checks.py", line 226, in contains_activation
raise Exception("Cannot detect activation type.")
Exception: Cannot detect activation type.
```
It is somehow confusing the LeakyRelu with Softmax activation and thus, fails to apply `model_wo_softmax` function on it.
Could you please help me fix this issue. I am stuck with this part since my model uses LeakyRelu activations. I would be grateful to you.
Thanks,
Naman
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.