mlcommons / mlcommons/endpoints

BFCL-v4 multi-turn runner: reuse HttpClientSampleIssuer instead of bespoke HTTP client

Open
#382 0 comments 1 reaction 1 assignee View on GitHub

@Palanivelg is already working on this.

Since Jul 22, 2026.

Dominant language
Python
Stars
21
Forks
28
Avg merge
3d 17h
Merged PRs (30d)
13

Description

Follow-up to #346 (review thread on src/inference_endpoint/evaluation/bfcl_v4_multi_turn_runner.py).

In #346, @arekay-nv and @viraatc noted that the BFCL-v4 multi-turn runner reimplements its own HTTP client and response parser instead of reusing the existing sample-issuer path.

Scope

  • Drive the multi-turn conversation loop through HTTPEndpointClient / HttpClientSampleIssuer, reusing the existing adapters and event logging.
  • Keep only BFCL-specific turn orchestration (custom tool execution between turns) in the runner.
  • Drop the bespoke httpx client and response parser from the runner.

Validation

  • Diff multi-turn accuracy scores before/after against a live server to confirm parity.

Notes

  • No functional change to the single-turn submission path; this is a refactor and does not block #346.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.