MedFailBench integration proposal for clinical AI safety evaluation

Hello OpenCompass team,

I am a physician from Turkey building MedFailBench, an open source clinical AI safety benchmark based on synthetic clinician authored cases.

Project: https://github.com/goktugozkanmd/medical-ai-failure-atlas

OpenCompass looks like the right technical route if MedFailBench should become a reusable clinical safety evaluation task rather than only a standalone repo.

I want to explore a China Turkey collaboration around multilingual clinical AI safety evaluation. A concrete first step could be a small MedFailBench adapter or task, followed by a shared model evaluation report and coauthored paper.

I can contribute the initial cases, safety taxonomy, scoring files, and integration work.

No patient data is involved. This is research infrastructure for safer medical LLM evaluation, not clinical advice or deployment.

Would your team be open to discussing a joint pilot?

Best,
Goktug Ozkan

Start by reviewing OpenCompass's task and benchmark integration conventions, then inspect the linked MedFailBench repository and its cases, safety taxonomy, and scoring files. Clarify the pilot's scope with the issue author before implementation. Done would mean an agreed adapter or task integrated into OpenCompass and a reproducible clinical safety evaluation.