benchopt / benchopt/benchmark_resnet_classif

[BUG] Initialization change for the dense layer in the TF implem of ResNet18 is not applied correctly

Open
#54 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
12
Forks
4
PR merge metrics
No merged PRs in 30d

Description

Indeed, because I am [changing the model itself](https://github.com/benchopt/benchmark_resnet_classif/blob/a6ed9d5dee7b1252e3d72d30735b099e46fa00cc/utils/tf_resnets.py#L239-L250), [iterating through the layers](https://github.com/benchopt/benchmark_resnet_classif/blob/a6ed9d5dee7b1252e3d72d30735b099e46fa00cc/utils/tf_resnets.py#L260) doesn't give me the last dense layer.

The change is therefore never applied when I remove the initial downsampling.

When I do not, there is the following bug:
```
Traceback (most recent call last):
File "/gpfsdswork/projects/rech/xpa/uap69lx/benchopt/benchopt/utils/pdb_helpers.py", line 28, in exception_handler
yield ctx
File "/gpfsdswork/projects/rech/xpa/uap69lx/benchopt/benchopt/runner.py", line 103, in run_one_to_cvg
solver.run(run_once_cb)
File "/gpfsdswork/projects/rech/xpa/uap69lx/benchmark_resnet_classif/utils/tf_solver.py", line 167, in run
self.model = self.model_init_fn()
File "/gpfsdswork/projects/rech/xpa/uap69lx/benchmark_resnet_classif/objective.py", line 120, in _model_init_fn
model = model_klass(
File "/gpfsdswork/projects/rech/xpa/uap69lx/benchmark_resnet_classif/utils/tf_resnets.py", line 331, in ResNet18
change_dense_init(model)
File "/gpfsdswork/projects/rech/xpa/uap69lx/benchmark_resnet_classif/utils/tf_resnets.py", line 264, in change_dense_init
layer.build(layer.input_spec.shape)
File "/gpfswork/rech/xpa/uap69lx/.local_torch/lib/python3.9/site-packages/keras/layers/core/dense.py", line 141, in build
raise ValueError('The last dimension of the inputs to a Dense layer '
ValueError: The last dimension of the inputs to a Dense layer should be defined. Found None. Full input shape received:
```

This is because the input specifications are not defined correctly despite the layers being built, I dk why.
I could simply base this shape on the current kernel shape.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in utils/tf_resnets.py at change_dense_init and ResNet18, especially the linked layer-iteration and dense-layer code. Reproduce model initialization with and without initial downsampling, then inspect the dense layer's kernel shape and input specification. Done means the dense initialization change is applied in both configurations without the undefined-input-shape error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.