Questions about implementation details to reproduce the results in the paper

Hi,

Thanks for creating DiscoveryBench. Wonderful work!

Recently, I’ve been trying to reproduce the results in your paper, and I’m facing several issues:

(1) It seems that in your codebase, you only provided the guideline to execute `discovery_agent.py` to generate a trajectory for one query, and `discovery_eval.py` to evaluate one single trajectory. Can you also provide the script you used to run all data examples in the real split, and how you calculated the overall score in your paper?

(2) Given that the output trajectory of discovery_agent.py is pure text, how did you implement the regex to extract the generated hypothesis and workflow summary? It would be really helpful if you could also provide your script for answer extraction, if any.

(3) Given the several example eval scripts you provided (such as `eval/example_eval_scripts/test_real_eval_3.sh`), I have no clue to find out how you get the gold_workflow, as well as how you extracted the gen_hypo and gen_workflow (as stated in (2)). A unified evaluation pipeline will really help for reproducing the results in your paper.

Looking forward to your reply. Thanks!