NVIDIA / NVIDIA/TileGym

Would cuTile ports of Liger's chunked preference losses (DPO, ORPO, SimPO, KTO) be accepted into the liger suite?

Open
#190 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
811
Forks
87
Avg merge
6d 4h
Merged PRs (30d)
9

Description

I would like to contribute cuTile ports of Liger-Kernel's chunked preference losses to src/tilegym/suites/liger/, following the grpo_loss fwd+bwd pattern and the chunked backward-in-forward structure of fused_linear_cross_entropy. Plan: one PR per loss, starting with DPO (LigerFusedLinearDPOLoss semantics: fused linear over chosen/rejected halves, per-sequence logps, reference-model support, the upstream loss_type variants).

Intended split, mirroring the FLCE port: torch/cuBLAS keeps the GEMMs, cuTile owns the two row-wise kernels (fused per-row logsumexp plus target log-prob, and the in-place d_logits write), and the O(n_pairs) preference formulas stay in device-side torch so every variant matches upstream exactly. Upstream implements these in chunked torch.compile rather than a handwritten kernel, so the benchmark baseline would be Liger's compiled path plus a naive full-logits torch reference, validated on Ampere or Blackwell.

Before I start: is the liger suite open to external ports? It is NVIDIA-authored so far, and I do not want to collide with internal plans. Two scope questions if yes:

  1. Full loss_type coverage in the first DPO PR, or sigmoid first with variants as follow-ups?
  2. Perf table in the PR description only, like the existing liger suite ops, or an in-repo benchmark file like tests/benchmark/suites/unsloth/?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing src/tilegym/suites/liger/ and the grpo_loss and fused_linear_cross_entropy patterns, then compare the upstream Liger chunked preference-loss implementations. Resolve whether external ports are accepted and whether the first DPO port covers all loss_type variants. Done requires an agreed scope, matching semantics, and benchmark validation on Ampere or Blackwell.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.