apple / apple/coremltools

Quantized Models Chunking into unequal sizes

Open
#2,320 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🐞Describing the bug
with reference to this issue https://github.com/apple/ml-stable-diffusion/issues/353, i used the bisect_model() function to split a quantized model into 2 chunks, i tried with 7.1 and 7.0 with reference to this file:https://github.com/apple/ml-stable-diffusion/blob/cf16df8207dfcba685a9391bad04f7402ea87b73/python_coreml_stable_diffusion/chunk_mlprogram.py#L123 , but was facing same issue.
```
prog = _load_prog_from_mlmodel(model)

# Compute the incision point by bisecting the program based on weights size
op_idx, first_chunk_weights_size, total_weights_size = _get_op_idx_split_location(
prog)
print(f"First chunk size = {first_chunk_weights_size:.2f} MB") # 152.67 MB
print(f"Second chunk size = {total_weights_size - first_chunk_weights_size:.2f} MB") #0.42 MB
print(index=587/2720)
prog_chunk1 = _make_first_chunk_prog(f"index={op_idx}/{len(main_block.operations)") # 587/3000
prog_chunk2 = _make_second_chunk_prog(_load_prog_from_mlmodel(model), op_idx)
```

## System environment (please complete the following information):
- coremltools version:8.0b2

cc: @aseemw

Contributor guide

Open the contributing guide

Research direction

Start with the referenced issue 353 and chunk_mlprogram.py around bisect_model() and _get_op_idx_split_location(). Reproduce the reported behavior with a quantized model using coremltools 8.0b2 and the 7.1/7.0 configurations described, then compare the calculated operation index and chunk weight sizes. Done means the cause of the unequal chunk sizes is identified and the splitting behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.