isl-org / isl-org/Open3D

Batched matrix multiplication via matmul

Open
#5,004 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
14k
Forks
2.6k
Avg merge
5d 18h
Merged PRs (30d)
6

Description

### Checklist

- [X] I have searched for [similar issues](https://github.com/isl-org/Open3D/issues).
- [X] For Python issues, I have tested with the [latest development wheel](http://www.open3d.org/docs/latest/getting_started.html#development-version-pip).
- [X] I have checked the [release documentation](http://www.open3d.org/docs/release/) and the [latest documentation](http://www.open3d.org/docs/latest/) (for `master` branch).

### Proposed new feature or change

Let 3D tensors A,B represent arrays of matrices; we want the Tensor::Matmul operation to also support matrix-multiplying every matrix in A with every corresponding matrix in B, similar to `C = numpy.matmul(A,B)` in numpy.

I wanted to post this issue just to avoid replication of work if someone decides to add this to Open3D later -- so they can use my code, since I've already done the bulk of the work for this in the `greg_devel` branch of the Neural Non-Rigid Tracking & Fusion repo.

Check the [linalg](https://github.com/Algomorph/NeuralTracking/tree/greg_devel/csrc/core/linalg) folder there (ignore LinalgHeaders*.h and LinalgUtils.* files -- these are identical to Open3D ones). Note that the `MKL` and `cuBLAS` implementations are there in `BlasWrapper.h`, but the OpenBLAS implementation is missing (due no convenient `cblas_?gemm_batch` / `cublas?gemmBatched` equivalents in OpenBLAS and my laziness in implementing a naive one with a for loop and `cblas_?gemm`). You'll find my take on device selection [here](https://github.com/Algomorph/NeuralTracking/blob/greg_devel/csrc/core/DeviceSelection.h). That and usage of templates to denote CPU/CUDA functions of things instead of preprocessor macros may need to be replaced by the current Open3D conventions to keep Open3D code consistent.

The other relevant files include top-level C++ library routine Matmul3D, [here](https://github.com/Algomorph/NeuralTracking/blob/greg_devel/csrc/core/TensorManipulationRoutines.h) and [here](https://github.com/Algomorph/NeuralTracking/blob/greg_devel/csrc/core/TensorManipulationRoutines.cpp). Instead of adding a new function, I recommend combining the functions from my `core/linalg/Matmul3D.*` to the existing functions in `core/linalg/Matmul.*` , and let the `Matmul` function decide whether to do a single or a batch multiplication based on the dimentions of the two input tensors.

### References

[Reference](https://numpy.org/doc/stable/reference/generated/numpy.matmul.html) for matmul routine in NumPy
[Reference](https://pytorch.org/docs/stable/generated/torch.matmul.html) for matmul routine in PyTorch

As you see, Open3D is the odd duck currently in making `Matmul` an instance function/method (as opposed to a namespace/module function) and, most importantly, on support of batched matrix multiplication in `Matmul`, hopefully this is convincing enough of its necessity.

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.