microsoft / microsoft/agent-framework

.NET: [Feature]: Expose the MEAI IEvaluator -> IAgentEvaluator bridge (AsAgentEvaluator)

Open
#7,363 1 comment 0 reactions 1 assignee Claimed by @chetantoshniwal View on GitHub
.NET
Dominant language
Python
Stars
13.6k
Forks
2.3k
Avg merge
2d 45m
Merged PRs (30d)
358

Description

### Description

`Microsoft.Agents.AI` already contains exactly the right bridge between the `Microsoft.Extensions.AI.Evaluation` ecosystem and agent evals — `MeaiEvaluatorAdapter` (`dotnet/src/Microsoft.Agents.AI/Evaluation/MeaiEvaluatorAdapter.cs`) wraps any MEAI `IEvaluator` into an `IAgentEvaluator`, runs it per item, handles the `RawResponse`-vs-bare-text shapes, and aggregates results. But it is `internal` (verified on `main` @ `7b6d25798`), so every consumer who wants to run an MEAI evaluator (the bundled quality evaluators, the safety evaluators, or their own `IEvaluator`) through `LocalEvaluator`-style agent pipelines must re-implement the same ~40-line adapter verbatim.

- **What problem does it solve?** Removes a copy-paste seam between the two first-party evaluation ecosystems. MEAI has a rich, growing `IEvaluator` catalog; agent-framework's eval pipeline speaks `IAgentEvaluator`. The bridge exists, is tested by usage, and is the obvious connector — it just isn't reachable.
- **Expected behavior:** any MEAI `IEvaluator` becomes an `IAgentEvaluator` in one call.
- **Alternatives considered:** keep copying the adapter into every consumer (status quo — drift risk per copy); make the class `public` without an extension method (works, but an `As*` extension matches the framework's existing `AsAIAgent` / MEAI `AsChatClient` idiom).

**Proposed API**

```csharp
namespace Microsoft.Agents.AI;

public static class AgentEvaluatorExtensions
{
/// Wraps an MEAI evaluator so it can run in agent evaluation pipelines.
public static IAgentEvaluator AsAgentEvaluator(
this IEvaluator evaluator,
ChatConfiguration chatConfiguration);
}
```

Implementation is a one-line delegation to the existing `MeaiEvaluatorAdapter`, which can stay `internal`. Purely additive — no behavior change for existing callers. Happy to send the PR (extension class + XML docs + one adapter test; no changes to `MeaiEvaluatorAdapter` itself) if the shape is acceptable.

### Code Sample

```csharp
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI.Evaluation;
using Microsoft.Extensions.AI.Evaluation.Quality;

IAgentEvaluator quality = new CoherenceEvaluator()
.AsAgentEvaluator(new ChatConfiguration(judgeChatClient));

AgentEvaluationResults results = await quality.EvaluateAsync(items, "quality-gate");
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.