mlcommons / mlcommons/endpoints
BFCL multi-turn: endpoint-controlled tool-call names reach bfcl-eval eval() (RCE on evaluator host)
@Palanivelg is already working on this.
Since Jul 22, 2026.
- Dominant language
- Python
- Stars
- 21
- Forks
- 28
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 13
Description
From PR #346 review (bfcl_v4_execution.py:235).
Decoded tool-call strings (function name + arguments straight from the endpoint's tool_calls response) reach bfcl-eval's eval(func_call), guarded only by an upstream name denylist (kill/exit/quit/remove/unlink/popen/Popen/run). A tool call named __import__('os').system bypasses it (split("(")[0] yields __import__, not on the list), so a compromised or untrusted --endpoint gets arbitrary code execution on the evaluator host.
This is upstream bfcl-eval behavior, but this PR is what wires arbitrary-endpoint HTTP responses into it.
Options:
- At minimum: document that multi-turn eval must only target trusted endpoints.
- Better: validate
tc["name"]against the entry's known tool names inprocess_responsebefore handing calls to bfcl-eval.
Deferred out of PR #346 to keep it focused; the multi-turn live-execution path is opt-in (its own CLI).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.