NVIDIA / NVIDIA/nvalchemi-toolkit

🐛[BUG]: Threshold is actually optional if using custom_op

Open
#83 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
167
Forks
38
Avg merge
8d 15h
Merged PRs (30d)
7

Description

Version

0.1.0

On which installation method(s) does this occur?

Source

Describe the issue

The ConvergenceTreshold requires a threshold even if it's not needed used because we're using a custom operation.

Should we either make it optional if custom_op is set or pass it as an arg to the custom_op? I'd suggest the latter so users can tweak the tolerance of a convergence hook more easily.

Minimum reproducible example
import torch
from nvalchemi.dynamics import ConvergenceHook


def _custom_noop(x: torch.Tensor) -> torch.Tensor:
    return torch.ones(1, dtype=torch.bool, device=x.device)

ConvergenceHook = ConvergenceHook(
    criteria={"key": "custom_metric", "custom_op": _custom_noop},
)
Relevant log output
Traceback (most recent call last):
  File "/mnt/c/Users/lward/Code/nv-open/nv-alchemi/nvalchemi-toolkit/scripts/thr-mre.py", line 8, in <module>
    ConvergenceHook = ConvergenceHook(
        criteria={"key": "custom_metric", "custom_op": _custom_noop},
    )
  File "/mnt/c/Users/lward/Code/nv-open/nv-alchemi/nvalchemi-toolkit/nvalchemi/dynamics/base.py", line 2288, in __init__
    self.criteria = [_ConvergenceCriterion(**criteria)]
                     ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/mnt/c/Users/lward/Code/nv-open/nv-alchemi/nvalchemi-toolkit/.venv/lib/python3.13/site-packages/pydantic/main.py", line 250, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for _ConvergenceCriterion
threshold
  Field required [type=missing, input_value={'key': 'custom_metric', ...noop at 0x7a7d06ee7100>}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.12/v/missing
Environment details

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in nvalchemi/dynamics/base.py at _ConvergenceCriterion and ConvergenceHook, then reproduce the provided example with the custom operation. Trace how threshold validation interacts with custom_op and determine which behavior the issue should adopt. Done means a custom_op criterion can be constructed as intended and regression coverage verifies the selected threshold behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.