OpenEuroLLM / OpenEuroLLM/Taskboard
Instruction-tuned current multilingual models
Open
@ezosa is already working on this.
Since Nov 10, 2025.
4.6 post-training
T4.3 - datamix/proc
T5.2 - dynamic evals
- Dominant language
- No language data
- Stars
- 3
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
We trained on multilingual mixes with various proportion for instance, 80%/20% english/multilingual:
- 100% english: https://huggingface.co/openeurollm/datamix-2b-en
- 90% english: https://huggingface.co/openeurollm/datamix-2b-90-10
- 80% english: https://huggingface.co/openeurollm/datamix-2b-80-20
We should also compare the following checkpoints: https://huggingface.co/open-sci/open-sci-ref-v0.01-1.7b-nemotron-hq-1T-4096
This task consists in:
- Running Tulu3 SFT and DPO on all the checkpoints above (or an equivalent pipeline in another framework that we know is good enough)
- Evaluating IFEval, HumanEval, Alpaca-Eval, Arena-Hard, m-Arena-Hard, GSM8K on the base and instruction tuned model
- Comparing with baselines on Mixture-vitae
- Plotting table & results in this issue
I am not putting an assignee yet, let me know if you want to work on this otherwise I will do it over the next two weeks.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.