linkedin / linkedin/Liger-Kernel
Need to investigate Gemma3 implementation with Liger
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.6k
- Forks
- 603
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 47
Description
🐛 Describe the bug
The tolerance when comparing loss in gemma3 multimodal model need to be set high (atol,rtol - 1e-3) compare to others (atol=1e-8,rtol=1e-5) in order to pass the tests.
Similar for gemma3_text, need to set (atol = 3e-1, rtol = 4e-1) to pass the tests when comparing top 20 log probs.
Reproduce
No response
Versions
Operating System: Linux-5.15.180.1-1.cm2-x86_64-with-glibc2.35
Python version: 3.10.14
Liger Kernel version: 0.5.10
PyTorch version: 2.7.1+cu126
CUDA version: 12.6
Triton version: 3.3.1
Transformers version: 4.52.4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the tests that compare Gemma3 multimodal loss and Gemma3 text top-20 log probabilities, then reproduce them in the reported Python, PyTorch, CUDA, Triton, and Transformers environment. Trace the Gemma3 implementation and comparison path to determine why the tolerances differ; done means the underlying discrepancy is explained and the relevant comparisons pass without unexplained tolerance increases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100