Request gold compile.sh scripts

Hi there.

Really like this benchmark. I’m building a Modal-based pipeline to evaluate ProgramBench with higher parallelism, and I’d like to validate my implementation end-to-end without running a full agent each time.

For that purpose, I’m looking for a way to mock the agent generation phase with known-good submissions. The repo part is simple as the commit hash is known. As for the compile script part, it is a little bit tricky. Would it be possible to release the compile.sh scripts used to build the gold/reference executables, or any equivalent reference build scripts?

I understand if these cannot be shared due to benchmark integrity concerns. In that case, is there a recommended way to create a small set of known-good mock submissions for validating the evaluation pipeline?

Individua gli script compile.sh di riferimento e analizza il modo in cui la pipeline di valutazione utilizza i commit del repository e gli eseguibili compilati. Determina se gli script possono essere rilasciati senza compromettere l’integrità del benchmark; il lavoro è completato fornendo gli script di build di riferimento oppure documentando un metodo supportato per creare mock submissions note come valide.

Scritto dal modello di indicizzazione a partire dal testo della issue.