[BUG] tune_with_strategy does not return config, metrics tuple as expected

Issue:

tune_with_strategy is a tuning method to try different configs and return metrics for each config. It returns the tuple, but first item is empty, second is metrics as expected.

How to reproduce:

1. Run notebook as in using_vasim.ipynb
2. Output results as

    results = tune_with_strategy(config_path, strategy, num_combinations=num_combinations,
                        num_workers=num_workers, data_dir=data_dir,
                        algorithm=selected_algorithm, initial_cpu_limit=initial_cpu_limit,
                        algo_specific_params_to_tune=algo_specific_params_to_tune,
                        general_params_to_tune=params_to_tune,
                        predictive_params_to_tune=predictive_params_to_tune)
   print(results)

Actual Output

[({}, {'average_slack': 1.701791893260048, 'average_insufficient_cpu': 0.025321080865735254, 'sum_slack': 19487.21896972081, 'sum_insufficient_cpu': 289.95169699353437, 'num_scalings': 287, 'num_insufficient_cpu': 225, 'insufficient_observations_percentage': 1.9648938957296307, 'slack_percentage': 16.188019629193107, 'median_insufficient_cpu': 0.0, 'median_slack': 1.4628571428571426, 'max_slack': 7.236000000000001}), ({}, {'average_slack': 1.7375966264711211, 'average_insufficient_cpu': 0.025321080865735254, 'sum_slack': 19897.21896972081, 'sum_insufficient_cpu': 289.95169699353437, 'num_scalings': 283, 'num_insufficient_cpu': 225, 'insufficient_observations_percentage': 1.9648938957296307, 'slack_percentage': 16.472503193314715, 'median_insufficient_cpu': 0.0, 'median_slack': 1.5, 'max_slack': 7.236000000000001}), ({}, {'average_slack': 2.6873225244687506, 'average_insufficient_cpu': 0.011724998250317914, 'sum_slack': 30772.530227691663, 'sum_insufficient_cpu': 134.26295496439045, 'num_scalings': 287, 'num_insufficient_cpu': 99, 'insufficient_observations_percentage': 0.8645533141210375, 'slack_percentage': 23.344090476660988, 'median_insufficient_cpu': 0.0, 'median_slack': 2.4628571428571426, 'max_slack': 8.236}), ({}, {'average_slack': 2.723127257679824, 'average_insufficient_cpu': 0.011724998250317914, 'sum_slack': 31182.530227691663, 'sum_insufficient_cpu': 134.26295496439045, 'num_scalings': 283, 'num_insufficient_cpu': 99, 'insufficient_observations_percentage': 0.8645533141210375, 'slack_percentage': 23.58177153529353, 'median_insufficient_cpu': 0.0, 'median_slack': 2.5, 'max_slack': 8.236}), ({}, {'average_slack': 3.6803884796175477, 'average_insufficient_cpu': 0.005664239574995363, 'sum_slack': 42144.12848010054, 'sum_insufficient_cpu': 64.8612073732719, 'num_scalings': 287, 'num_insufficient_cpu': 50, 'insufficient_observations_percentage': 0.43664308793991796, 'slack_percentage': 29.417418012460022, 'median_insufficient_cpu': 0.0, 'median_slack': 3.460000000000001, 'max_slack': 9.236}), ({}, {'average_slack': 3.716193212828621, 'average_insufficient_cpu': 0.005664239574995363, 'sum_slack': 42554.12848010054, 'sum_insufficient_cpu': 64.8612073732719, 'num_scalings': 283, 'num_insufficient_cpu': 50, 'insufficient_observations_percentage': 0.43664308793991796, 'slack_percentage': 29.61884040446191, 'median_insufficient_cpu': 0.0, 'median_slack': 3.5, 'max_slack': 9.236})]

Expected output

list of tuples with non-empty config (dict) and metrics

Exit criteria

Fix and unit test

Start with tune_with_strategy and reproduce the empty configuration results using using_vasim.ipynb. Trace how each tuning configuration is passed into the returned tuple, then add a unit test covering the expected non-empty configuration dictionary and metrics. Done means the method returns both values for each result and the test passes.