NVIDIA / NVIDIA/NeMo-Agent-Toolkit

Add phase-specific field selection for middleware on multi-field boundary objects

Open
#2,098 2 comments 1 reaction 1 assignee View on GitHub

@ericevans-nv is already working on this.

Since Jun 30, 2026.

improvement
Dominant language
Python
Stars
2.6k
Forks
762
Avg merge
21h 28m
Merged PRs (30d)
27

Description

Summary

Middleware can run at two invocation phases: before a function runs (pre_invoke) and after a function returns (post_invoke). For simple single-value functions, one selected value may be sufficient for both phases. However, for stateful or multi-field boundary objects, the same object can contain multiple semantic fields: request-side fields, response-side fields, history, intermediate state, or generated output.

Today, dynamic middleware field selection defines one set of fields per wrapped function, and that same selection is reused for both pre_invoke and post_invoke. This becomes ambiguous when the selected boundary object contains multiple fields with different phase semantics.

Problem

A single shared field selection can cause middleware to evaluate the wrong fields for a phase.

For example, a function may receive or return a state object shaped like:

state:
  request_items:
    - text
  response_items:
    - text

With one shared field selection:

workflow_functions:
  some_function:
    state:
      request_items:
        - text
      response_items:
        - text

both phases use both fields:

  • pre_invoke evaluates both request-side and response-side fields
  • post_invoke evaluates both request-side and response-side fields

This is not an issue for a single string argument, but it is an issue for multi-field state objects where fields have different meanings depending on phase.

Desired behavior

Field selection should be expressible per middleware phase for multi-field boundary objects, for example:

workflow_functions:
  some_function:
    pre_invoke:
      state:
        request_items:
          - text
    post_invoke:
      state:
        response_items:
          - text

Equivalent naming such as input / output or pre / post could also work. The important capability is that middleware can choose different traversal targets for each invocation phase.

Semantics

pre_invoke selection should define which fields middleware evaluates before the wrapped function runs. These fields usually represent input, request, or user-controlled content entering the function boundary.

post_invoke selection should define which fields middleware evaluates after the wrapped function returns. These fields usually represent output, response, or generated content leaving the function boundary.

If only the existing shared field selection is configured, middleware should preserve current behavior for backward compatibility.

Motivating use case

Guardrails middleware exposes this issue with agent state objects that contain conversation history. A state may include both user/request messages and assistant/response messages. Input rails should evaluate user/request content entering the function, while output rails should evaluate assistant/response content produced by the function.

With shared selection, output rails can accidentally evaluate original input fields. For example, if a user prompt contains a sensitive-looking literal and the assistant safely avoids repeating it, the output rail can still block the safe response because it evaluated the original request field during post_invoke.

Acceptance criteria
  • Dynamic middleware config can express separate field selections for pre_invoke and post_invoke.
  • pre_invoke uses phase-specific fields when configured.
  • post_invoke uses phase-specific fields when configured.
  • Existing shared field-selection config remains backward compatible.
  • Tests cover a multi-field boundary object containing both request-side and response-side string fields, verifying post_invoke does not evaluate request-side fields when a post-specific selection is configured.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.