2024 Q4 Roadmap

As the community grows, keeping track of issues and PRs becomes more and more challenging. This pinned issue will serve as the central place to manage the progress in 2024 Q4 (~2024/12). Here we only list the important and top level issues/PRs.

# Progress Tracker

Format: [title] [link], [contributor] / [reviewer (maintainer)]

## New Features: Liger to Post Training

https://github.com/linkedin/Liger-Kernel/issues/371

## Models

- llama 3.2 vision (https://github.com/linkedin/Liger-Kernel/pull/282), @tyler-romero / @shivam15s
- pixtral 12B (https://github.com/linkedin/Liger-Kernel/pull/253), @AndreSlavescu / @ByronHsu
- Flux model (https://github.com/linkedin/Liger-Kernel/issues/73), call for help / @qingquansong
- DeepseekV2 (https://github.com/linkedin/Liger-Kernel/issues/129), call for help / @qingquansong
- Gemma2 (https://github.com/linkedin/Liger-Kernel/issues/127), call for help / @yundai424
- Qwen2-VL (https://github.com/linkedin/Liger-Kernel/pull/283), @tyler-romero / @shivam15s
- Jamba (https://github.com/linkedin/Liger-Kernel/pull/214), @yubofredwang / @ByronHsu

## Kernels

- TVD loss, https://github.com/linkedin/Liger-Kernel/pull/324 / @qingquansong @lancerts
- JSD loss, https://github.com/linkedin/Liger-Kernel/pull/300 / @qingquansong @lancerts
- GroupNorm (https://github.com/linkedin/Liger-Kernel/pull/225), @denti / @shivam15s
- Z Loss in cross entropy (https://github.com/linkedin/Liger-Kernel/pull/239), @Tcc0403 / @shivam15s
- Flash Attention in Triton (https://github.com/linkedin/Liger-Kernel/pull/275), @remi-or / @shivam15s
- Conv2d (https://github.com/linkedin/Liger-Kernel/pull/228), @AndreSlavescu / @lancerts
- Triton mm int8 x int2(https://github.com/linkedin/Liger-Kernel/pull/195), @mekkcyber / @ByronHsu

## Testing

- `assert_verbose_allclose` (https://github.com/linkedin/Liger-Kernel/pull/261), @Tcc0403 / @ByronHsu

## Patching

- Weights are not copied for instance patching (https://github.com/linkedin/Liger-Kernel/pull/280), @shimizust / @ByronHsu

# Community Sync

TBD

Start by reviewing the linked issues and pull requests in the Progress Tracker, then compare their current statuses with the entries here. Done means the tracker accurately reflects active work and the Community Sync section has been resolved or clearly updated.

Written by the indexing model from the issue text.