Owner

CarperAI

11 indexed repositories · View on GitHub

trlx
A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

Python · 4754 stars
OpenELM
Evolution Through Large Models

Python · 746 stars
cheese
Used for adaptive human in the loop evaluation of language and embedding models.

Python · 306 stars
DRLX
Diffusion Reinforcement Learning Library

Python · 193 stars
Code-Pile
This repository contains all the code for collecting large scale amounts of code from GitHub.

Python · 110 stars
autocrit
A repository for transformer critique learning and generation

Python · 87 stars
InstructGPT
For experiments involving instruct gpt. Currently used for documenting open research questions.

71 stars
squeakily
A library for squeakily cleaning and filtering language datasets.

Jupyter Notebook · 50 stars
Algorithm-Distillation-RLHF

Python · 35 stars
magicarp-v2
magiCARP is an API used for crossencoder training.

Python · 9 stars
AutoPaperclipMaximizer
👀

3 stars

Open All

Add proper citation of deduplication Open

CarperAI/squeakily#9 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Moving away from diffusers Open

CarperAI/magicarp-v2#16 · 0 comments · 0 reactions · 2 assignees · Indexed Sep 14, 2026
Mixed precision training Open

CarperAI/magicarp-v2#17 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 14, 2026
Simplify validation loader in pipeline Open

CarperAI/magicarp-v2#18 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
MidU Timestep tests Open

CarperAI/magicarp-v2#20 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Documentation with readthedocs Open

documentation

CarperAI/OpenELM#21 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Fix DDP in Codegen utils Open

CarperAI/OpenELM#31 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Automatic Prompt Engineering for Prompt based mutation Open

CarperAI/OpenELM#42 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
will the examples(as shown in the paper https://arxiv.org/abs/2302.12170) code added here ? Open

CarperAI/OpenELM#75 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
do I have some news for you guys... Open

CarperAI/OpenELM#76 · 0 comments · 2 reactions · 0 assignees · Indexed Sep 14, 2026
Access to training checkpoints Open

CarperAI/OpenELM#77 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Reproduce Constitutional AI Steps Open

CarperAI/autocrit#2 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Eval Instruct Open

CarperAI/autocrit#3 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
License Open

CarperAI/autocrit#20 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Gradient metrics Open

feature request

CarperAI/trlx#27 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Self Play Open

feature request

CarperAI/trlx#28 · 3 comments · 1 reaction · 0 assignees · Indexed Sep 14, 2026
Add Jax support Open

feature request

CarperAI/trlx#54 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
How we can improve the documentation for beginners Open

documentation

CarperAI/trlx#65 · 6 comments · 5 reactions · 0 assignees · Indexed Sep 14, 2026
How to attribute reward to multiple model runs in the same trajectory with PPO Open

feature request

CarperAI/trlx#69 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
How to attribute different rewards to parts of the same rollout with PPO? Open

feature request

CarperAI/trlx#79 · 5 comments · 0 reactions · 1 assignee · Indexed Sep 14, 2026
Best-of-n sampling for rollouts Open

feature request

CarperAI/trlx#166 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 14, 2026
T5 FLAN 11B Config for a consumer GPU Open

feature request

CarperAI/trlx#173 · 3 comments · 7 reactions · 0 assignees · Indexed Sep 14, 2026
Improve Documentation Process Open

feature request

CarperAI/trlx#176 · 0 comments · 2 reactions · 0 assignees · Indexed Sep 14, 2026
Add CD to publish PyPi Package Open

feature request

CarperAI/trlx#199 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
PPO efficiency improvement by not recomputing the advantage in Trainer Open

bug

CarperAI/trlx#202 · 4 comments · 3 reactions · 0 assignees · Indexed Sep 14, 2026
Remove yaml config files Open

feature request

CarperAI/trlx#223 · 4 comments · 7 reactions · 1 assignee · Indexed Sep 14, 2026
ModelConfig.tokenizer_path and TrainConfig.save_best Open

documentation

CarperAI/trlx#232 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Add Speculative Sampling Open

feature request research

CarperAI/trlx#273 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 14, 2026
Remove dynamic imports Open

feature request

CarperAI/trlx#277 · 4 comments · 2 reactions · 1 assignee · Indexed Sep 14, 2026
Add tensor padding options to improve hardware utilization Open

contributions welcome feature request good first issue

CarperAI/trlx#284 · 8 comments · 2 reactions · 0 assignees · Indexed Sep 14, 2026
Give a way to pass data that doesn't involve loading the whole prompt dataset to memory Open

feature request

CarperAI/trlx#304 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Colab + Accelerate Integration issue Open

bug

CarperAI/trlx#317 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Collapse reference+learner hydra heads when using LoRa Open

contributions welcome feature request good first issue

CarperAI/trlx#320 · 6 comments · 2 reactions · 0 assignees · Indexed Sep 14, 2026
TPU Integration Open

feature request

CarperAI/trlx#323 · 3 comments · 1 reaction · 1 assignee · Indexed Sep 14, 2026
RL related issues, code understanding , scores, rewards,ppo Open

feature request

CarperAI/trlx#326 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Generic Flan-T5 RL training Open

CarperAI/trlx#327 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
SHP reward model Open

feature request

CarperAI/trlx#336 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 14, 2026
Minimum Risk Training support Open

feature request

CarperAI/trlx#354 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Add support XGLM model Open

feature request

CarperAI/trlx#379 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 14, 2026
Flant-t5-large Deepspeed OVERFLOW! issues + bad outputs after trlx training Open

bug

CarperAI/trlx#383 · 8 comments · 1 reaction · 0 assignees · Indexed Sep 14, 2026
@LouisCastricato Yes, the idea is to give a fully reproducible script for the anyone to train it by themselves and validate. Open

CarperAI/trlx#393 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 14, 2026
!deepspeed examples/summarize_rlhf/sft/train_gptj_summarize.py is failing Open

CarperAI/trlx#410 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Question about trainer.save_pretrained Open

feature request

CarperAI/trlx#412 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Well-known RMs and eval utilities Open

feature request

CarperAI/trlx#415 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Confused by config parameters among total_steps, epochs, batch_size and num_rollouts Open

documentation

CarperAI/trlx#425 · 0 comments · 5 reactions · 0 assignees · Indexed Sep 14, 2026
Training problem for the GPT-j RLHF example Open

CarperAI/trlx#428 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Migrate to `peft` from `opendelta` for parameter efficient tuning methods Open

feature request

CarperAI/trlx#434 · 6 comments · 2 reactions · 0 assignees · Indexed Sep 14, 2026
FLAN-T5-small model run sentiments.ipynb demo gives so so many ""using bos_token ,but it is not set yet" lines" Open

bug

CarperAI/trlx#436 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
Loss should be averaged over all samples instead of tokens Open

bug

CarperAI/trlx#452 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 14, 2026
reproduce trlx example on wandb Open

bug

CarperAI/trlx#462 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 14, 2026

Showing the newest 100

This page lists what was indexed most recently. Advanced filter has the whole inventory, narrowed by language, difficulty and how long a task takes.

Open advanced filter

CarperAI

Showing the newest 100

Get new issues in your inbox