torch.arange free var issue
- Dominant language
- C++
- Stars
- 23
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
I noticed while compiling model that I was getting RAF free var. Using the IR_DUMP Python frames, we were able to trace the free var to be caused be torch.arange op.
In my bert_layer.py when I create position_ids for embedding like this:
```
position_ids = torch.arange(
seq_length, dtype=torch.long, device=
input_ids.device)
```
I get free var issue. However, this is simply resolved by doing this instead:
```
position_ids = torch.arange(
seq_length, dtype=torch.long
).to(input_ids.device)
```
I think the solution to this might be to implemented arange op & provide lowering to raf.arange in ratex. But I did not confirm as arange op does not show in LTC IR. Regardless, in case anyone sees similar issue, this would be a good reference.
Contributor guide
Research direction
Reproduce the BERT position_ids case from the issue, comparing torch.arange with device placement against the workaround that calls .to(input_ids.device). Use the IR_DUMP Python frames and inspect the LTC IR to determine how arange is represented; done means the direct torch.arange form compiles without the RAF free-variable issue and has the appropriate ratex lowering.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100