OpenPipe / OpenPipe/ART

Always Block when train

Open
#401 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
10.8k
Forks
989
Avg merge
6h 29m
Merged PRs (30d)
85

Description

[train.py] _calculate_logprobs - Processing chunk 2168 to 2176
[train.py] _calculate_logprobs - Processing chunk 2176 to 2184
[train.py] _calculate_logprobs - Processing chunk 2184 to 2192
[train.py] _calculate_logprobs - Processing chunk 2192 to 2200
[train.py] _calculate_logprobs - Processing chunk 2200 to 2208
[train.py] _calculate_logprobs - Processing chunk 2208 to 2216
[train.py] _calculate_logprobs - Processing chunk 2216 to 2224
[train.py] _calculate_logprobs - Processing chunk 10152 to 10160
[train.py] _calculate_logprobs - Processing chunk 10160 to 10168
[train.py] _calculate_logprobs - Processing chunk 10168 to 10176
[train.py] _calculate_logprobs - Processing chunk 10176 to 10184
[train.py] _calculate_logprobs - Processing chunk 10184 to 10192
[train.py] calculate_logprobs - Processing chunk 10192 to 10200
[train.py] calculate_logprobs - Processing chunk 10200 to 10208
[train.py] calculate_logprobs - Processing chunk 10208 to 10216
[train.py] calculate_logprobs - Processing chunk 10216 to 10224
==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1
\ /| Num examples = 10,000,000 | Num Epochs = 3 | Total steps = 60,000,000
O^O/ _/ \ Batch size per device = 1 | Gradient accumulation steps = 1
\ / Data Parallel GPUs = 1 | Total batch size (1 x 1 x 1) = 1
"-
-" Trainable parameters = 20,185,088 of 7,635,801,600 (0.26% trained)

DEBUG:urllib3.connectionpool:Resetting dropped connection: stats.vllm.ai
DEBUG:urllib3.connectionpool:https://stats.vllm.ai:443 "POST / HTTP/1.1" 200 None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in train.py, particularly _calculate_logprobs, and inspect the logged chunk processing around the training run. Reproduce the reported blocking behavior and determine where progress stops; the issue does not specify a test or expected behavior beyond avoiding the block.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.