dmlc / dmlc/dlpack

How to share data without requiring consumer to "own" the input tensor?

Open
#123 4 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
C++
Stars
1.2k
Forks
169
PR merge metrics
No merged PRs in 30d

Description

I would like to have pytorch code call a subroutine that takes uses DLPack, such that the subroutine is generic across frameworks. However, I noticed that the interface provided by DLPack allows compliant interfaces to output only DLManagedTensor, not DLTensor, requiring the consumer subroutine to take ownership of the input. Indeed, the [documentation](https://dmlc.github.io/dlpack/latest/python_spec.html) says: "The consumer must transer ownership of the DLManangedTensor from the capsule to its own object."

This is no good if you want the input tensor to continue to be used after the subroutine. Here is a small example to describe what I want to do:

```
my_model = Model()
output = my_model(input)
result = foreign_library.function_accepting_dl_tensor(output.detach())
print(output) # Accessing deleted memory
```

Is DLPack just no good for this use case? `__dlpack__()` outputs a DLManagedTensor (well, a capsule refering to a DLManagedTensor), which forces the consumer to take ownership (i.e., it implements what C++ programmers call "move semantics").

I suppose one work around might be:

```
my_model = Model()
output = my_model(input)
output_managed_tensor = output.detach().__dlpack__()
result = foreign_library.function_accepting_dl_tensor(output_managed_tensor)
output2 = torch.from_dlpack(output_managed_tensor)
print(output2) # No longer accessing deleted memory
```

However, this causes the chain of deleters (the deleter function pointer has to call the original function pointer) to grow every time this rigamarole happens, so it seems very non ideal.

Is there basically no way to access just a DLTensor instead of a DLManagedTensor if I want reference semantics instead of move semantics?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.