apple / apple/coremltools

Convert BetterTranformer model - no support for _nested_tensor_from_mask

Open
#1,777 0 comments 4 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

Hi,

I optimized a huggingface model with the BetterTransformer framework using [huggingface optimum.](https://huggingface.co/docs/optimum/bettertransformer/overview)
Here the simple code snippet:
```
from transformers import AutoModelForSequenceClassification
from optimum.bettertransformer import BetterTransformer
newModel = BetterTransformer.transform(baseline_model, keep_original_model=True)
```
BetterTransformer avoids unnecessary processing of padding tokens with nested tensors.

When I try to convert the optimized model I get the following error:

`_PyTorch convert function for op '_nested_tensor_from_mask' not implemented._`

Is there any way to fix the error? I haven't found any approach to implement the '_nested_tensor_from_mask' operation accordingly yet.

Thanks in advance!

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure using the BetterTransformer conversion snippet and the model-conversion path described in the issue. Start by locating how coremltools handles the reported `_nested_tensor_from_mask` operation; done means the optimized model converts without the reported unsupported-operation error and the behavior is covered by an appropriate conversion check.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.