automl / automl/NASLib

Storing of models or architectures in optimizers

Open
#134 1 comment 0 reactions 2 assignees Claimed by @abhash-er View on GitHub
zero cost merge
Dominant language
Python
Stars
595
Forks
133
PR merge metrics
No merged PRs in 30d

Description

In the `get_candidates` function of the `zerocost` branch optimizers `Bananas` and `Npenas` there is a discrepancy between how candidates in the `next_batch` are stored.

If the acquisition function is being optimized via `"random_sampling"`, then `model` is being stored:

```python
...
candidates.append(model)
```

Otherwise, if it is being optimized via `"mutation"`, then `model.arch` is being stored:

```python
...
candidate = arch
candidates.append(candidate)
```

However, the function `get_best_candidates` (which is called directly after `get_new_candidates`) treats `candidates` as a list of models:

```python
values = [acq_fn(model.arch, [{'zero_cost_scores' : model.zc_scores}]) for model in candidates]
```

Does this imply that the optimization of the acquisition function via `"mutation"` is not used in the main loop of either `Bananas` or `Npenas`? If so, how and when should this option be used?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.