huggingface / huggingface/ember

PyTorch and CoreML outputs do not match within 0.005, or PSNR < 35Max diff

Open
#5 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
20
Forks
5
PR merge metrics
No merged PRs in 30d

Description

Thanks for another great contribution!

However, for larger models (the one below as well as BAAI/bge-large-en-v1.5), I'm getting this error:

```
% uv run ember create
? Choose a model to export: intfloat/multilingual-e5-large - ~560M parameter model.
Model max sequence length: 512
`SentenceTransformer._target_device` has been deprecated, please use `SentenceTransformer.device` instead.
`SentenceTransformer._target_device` has been deprecated, please use `SentenceTransformer.device` instead.
`loss_type=None` was set in the config but it is unrecognised.Using the default loss: `ForCausalLMLoss`.
`SentenceTransformer._target_device` has been deprecated, please use `SentenceTransformer.device` instead.
`SentenceTransformer._target_device` has been deprecated, please use `SentenceTransformer.device` instead.
Converting PyTorch Frontend ==> MIL Ops: 0%| | 0/1027 [00:00 MIL Ops: 100%|████████████████████████████████████████████████████████████████████████████████████████████▉| 1026/1027 [00:00<00:00, 6926.50 ops/s]
Running MIL frontend_pytorch pipeline: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 192.52 passes/s]
Running MIL default pipeline: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 89/89 [00:06<00:00, 14.03 passes/s]
Running MIL backend_mlprogram pipeline: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 222.95 passes/s]
Traceback (most recent call last):
File "/Users/drbob/Development/ember/.venv/bin/ember", line 10, in
sys.exit(main())
^^^^^^
File "/Users/drbob/Development/ember/ember/cli.py", line 112, in main
create(
File "/Users/drbob/Development/ember/ember/create.py", line 299, in create
validate_model(
File "/Users/drbob/Development/ember/ember/create.py", line 198, in validate_model
raise ValueError(
ValueError: Model validation failed. PyTorch and CoreML outputs do not match within 0.005, or PSNR < 35Max diff: 0.00818544439971447, Avg diff: 0.0003821934515144676, PSNR: 50.24814224243164
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.