dotnet / dotnet/extensions

[API Proposal]: Allow MessageCountingChatReducer to retain function messages

Open
#7,756 0 comments 2 reactions 0 assignees View on GitHub
api-suggestion untriaged
Dominant language
C#
Stars
3.2k
Forks
894
Avg merge
1d 12h
Merged PRs (30d)
23

Description

### Background and motivation

`MessageCountingChatReducer` currently excludes every `ChatMessage` containing either `FunctionCallContent` or `FunctionResultContent` from the reduced output:

> Messages containing function call or function result content are excluded from the reduced output.

This behavior is useful when tool interactions should not be sent back to the model. However, some applications need to preserve function calls and their results when reducing chat history. For example, an agent may require previous tool results to answer follow-up questions correctly or to maintain the context of an earlier tool interaction.

The proposed API adds a Boolean constructor parameter that allows callers to retain messages containing function-related content.

The existing constructor and its current behavior are preserved for source and binary compatibility. Function messages are excluded by default and are retained only when callers explicitly opt in.

When retained, function messages are treated like other non-system messages and count toward `targetCount`.

### API Proposal

```csharp
namespace Microsoft.Extensions.AI;

public sealed class MessageCountingChatReducer : IChatReducer
{
public MessageCountingChatReducer(int targetCount);

public MessageCountingChatReducer(
int targetCount,
bool retainFunctionMessages);
}
```

The existing constructor behaves as if `retainFunctionMessages` were `false`.

When `retainFunctionMessages` is:

- `false`, messages containing `FunctionCallContent` or `FunctionResultContent` are excluded, preserving the current behavior.
- `true`, messages containing `FunctionCallContent` or `FunctionResultContent` are retained and count toward `targetCount`.

The option applies to the containing `ChatMessage`. It does not filter or modify individual entries in `ChatMessage.Contents`.

### API Usage

Preserve the current behavior and exclude function messages:

```csharp
var reducer = new MessageCountingChatReducer(targetCount: 20);
```

Explicitly retain messages containing function calls or function results:

```csharp
var reducer = new MessageCountingChatReducer(
targetCount: 20,
retainFunctionMessages: true);
```

Use the reducer in an agent chat history provider while retaining function messages:

```csharp
var chatHistoryProvider = new InMemoryChatHistoryProvider(new()
{
ChatReducer = new MessageCountingChatReducer(
targetCount: 20,
retainFunctionMessages: true),
ReducerTriggerEvent =
InMemoryChatHistoryProviderOptions.ChatReducerTriggerEvent.AfterMessageAdded,
});
```

### Alternative Designs

### An optional parameter on the existing constructor

```csharp
public MessageCountingChatReducer(
int targetCount,
bool retainFunctionMessages = false);
```

This API would be simpler at the source level. However, replacing the current constructor would remove the existing `.ctor(int)` from the assembly metadata. Applications compiled against an earlier version would continue looking for that constructor and could fail at runtime.

Keeping the existing constructor and adding an overload preserves both source and binary compatibility.

### A flags enum

A flags enum could allow function call messages and function result messages to be configured independently. This provides more flexibility but makes the API and its behavior more complex. The proposed Boolean addresses the primary scenario: retaining or excluding the complete function interaction.

### An options class

An options class would make it easier to introduce additional settings later, but it appears unnecessarily complex for a single Boolean choice.

### Changing the default behavior

The reducer could retain function messages by default. This would be a behavioral breaking change and could unexpectedly increase context usage for existing applications.

### Implementing a custom reducer

Applications can implement `IChatReducer` to obtain this behavior today. However, doing so requires duplicating the counting, system-message preservation, and ordering logic already provided by `MessageCountingChatReducer`.

### Risks

The proposal preserves the existing single-parameter constructor and its current behavior, avoiding source, binary, and behavioral breaking changes for existing consumers.

Retaining function messages can increase the amount of function-related content included in the reduced history. These messages count toward `targetCount`, keeping the reducer's size limit consistent and predictable.

A retained message may contain both function-related and non-function content. Since the option applies to the entire `ChatMessage`, all of its contents are retained without modification.

Contributor guide

Open the contributing guide

Research direction

Start by locating MessageCountingChatReducer and its IChatReducer implementation, then inspect the existing reduction behavior around ChatMessage, FunctionCallContent, and FunctionResultContent. Verify that the existing constructor still excludes function messages, while the proposed overload retains whole function-containing messages and counts them toward targetCount. Done means both constructor behaviors and source and binary compatibility are preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
api, backend-api-design
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.