llnl / llnl/Xscope

Code Bug Causes Incorrect Results

Open
#2 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2
Forks
3
PR merge metrics
No merged PRs in 30d

Description

We have identified a code bug in Xscope that leads to false positives for certain functions. Specifically, Xscope erroneously reports -INF exceptions for the exp function, which contradicts its mathematical properties. Through analysis, we determined the root cause lies in the improper mixing of fitness functions (used to guide the search) with the original function implementations. Below are the problematic code sections:

File: bo_analysis.py, Lines 229-231

# Function goals: (1) minimize (2) find INF (3) use fp inputs
def function_min_inf_fp_3(x0, x1, x2):
    return -call_GPU_kernel_3(x0, x1, x2)

The function_min_inf_fp_3 negates the result of the original function (call_GPU_kernel_3) and passes this modified function as func to the run_optimizer below.

File: bo_analysis.py, Lines 677-706

def run_optimizer(bounds, func, exp_name):
    global trials_to_trigger, trials_so_far
    trials_so_far = 0
    trials_to_trigger = -1
    if are_we_done(func, 0.0, exp_name):
        return
    optimizer = BayesianOptimization(f=func, pbounds=bounds, verbose=2, random_state=1)
    try:
        if verbose: print('BO opt...')
        utility = UtilityFunction(kind="ei", kappa=2.5, xi=0.1e-1)
        #utility = UtilityFunction(kind="ucb", kappa=10, xi=0.1e-1)
        #utility = UtilityFunction(kind="poi", kappa=10, xi=1e-1)
        for _ in range(bo_iterations):
            trials_so_far += 1
            next_point = optimizer.suggest(utility)
            target = func(**next_point)
            optimizer.register(params=next_point, target=target)

            update_runs_table(exp_name)

            # Check if we are done
            if are_we_done(func, target, exp_name):
                return
    except Exception as e:
        if verbose: print("Oops!", e.__class__, "occurred.")
        if verbose: print(e)
        if verbose: logging.exception("Something awful happened!")
    if verbose: print(optimizer.max)
    val = optimizer.max['target']
    save_results(val, exp_name)

The value optimizer.max['target'] directly uses the result from the modified function (function_min_inf_fp_3), rather than the original call_GPU_kernel_3, leading to false positives.

Impact: This bug causes invalid -INF reports for functions like exp, which should never produce such values under normal mathematical conditions.

Proposed Fix: Ensure the optimizer evaluates and saves results using the original function (call_GPU_kernel_3) instead of the negated version.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in bo_analysis.py with function_min_inf_fp_3 at lines 229-231 and run_optimizer at lines 677-706. Trace how the negated function is evaluated, registered, and saved, then verify that exp no longer produces invalid -INF reports and that results use call_GPU_kernel_3.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.