Tensor::SizeInBytes() returns incorrect size for empty strided tensors

Describe the bug

Empty strided tensors have storage of size of 1 element, yet Tensor::SizeInBytes returns number of elements * sizeof(type of tensor).

>>> t = torch.empty_strided((128,128), (0,0))
>>> t.storage()
-1884438375956480.0
[torch.FloatStorage of size 1]
>>> t.to(ort_device)

onnxruntime::Tensor::SizeInBytes() returns 65536 (128*128*sizeof(float)) at frame 4; however, , src.storage().nbytes() returns 4 at frame 13. Since PyTorch gives the storage size in the public API, could we update onnxruntime::tensor::SizeInBytes() to return the true storage size? The EP needs to know the size in bytes of the CPU tensor to copy the data to/from the device in question.

#1  0x00007f1a98ed6bf7 in onnxruntime::IDataTransfer::CopyTensor (this=0x6dbab90, src=..., dst=...) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/onnxruntime.git/onnxruntime/core/framework/data_transfer.cc:21
#2  0x00007f1a98472877 in onnxruntime::ProviderHostImpl::IDataTransfer__CopyTensor (this=0x7f1a9a0494e0 <onnxruntime::provider_host_>, p=0x6dbab90, src=..., dst=...)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/onnxruntime.git/onnxruntime/core/session/provider_bridge_ort.cc:500
#3  0x00007f1ab690401e in onnxruntime::IDataTransfer::CopyTensor (this=0x6dbab90, src=..., dst=...) at /home/ashari/ApolloOnnxRT/src/ExecutionProvider/provider_bridge_provider.cc:323
#4  0x00007f1a9832057a in torch_ort::eager::copy (invoker=..., src=..., dst=...) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/onnxruntime.git/orttraining/orttraining/eager/ort_ops.cpp:19
#5  0x00007f1a9824074f in torch_ort::eager::aten::copy_ (self=..., src=..., non_blocking=false) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/onnxruntime.git/orttraining/orttraining/eager/ort_aten.cpp:455
#6  0x00007f1a982ac3fa in c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor& (at::Tensor&, at::Tensor const&, bool), &torch_ort::eager::aten::copy_>, at::Tensor&, c10::guts::typelist::typelist<at::Tensor&, at::Tensor const&, bool> >::operator()(at::Tensor&, at::Tensor const&, bool) (this=0x4d299f0, args=false, args=false, args=false)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/build/lib.linux-x86_64-3.8/torch/include/ATen/core/boxing/impl/WrapFunctionIntoFunctor.h:13
#7  c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor& (at::Tensor&, at::Tensor const&, bool), &torch_ort::eager::aten::copy_>, at::Tensor&, c10::guts::typelist::typelist<at::Tensor&, at::Tensor const&, bool> >, at::Tensor& (at::Tensor&, at::Tensor const&, bool)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor&, at::Tensor const&, bool) (functor=0x4d299f0, args=false, args=false, args=false)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/build/lib.linux-x86_64-3.8/torch/include/ATen/core/boxing/impl/make_boxed_from_unboxed_functor.h:424
#8  0x00007f1abbf2dab7 in c10::callUnboxedKernelFunction<at::Tensor&, at::Tensor&, at::Tensor const&, bool> (
    unboxed_kernel_func=0x7f1a982ac340 <c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor& (at::Tensor&, at::Tensor const&, bool), &torch_ort::eager::aten::copy_>, at::Tensor&, c10::guts::typelist::typelist<at::Tensor&, at::Tensor const&, bool> >, at::Tensor& (at::Tensor&, at::Tensor const&, bool)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor&, at::Tensor const&, bool)>, functor=0x4d299f0, dispatchKeySet=..., args=@0x7ffd781f36df: false,
    args=@0x7ffd781f36df: false, args=@0x7ffd781f36df: false) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/boxing/KernelFunction_impl.h:57
#9  0x00007f1abc441c33 in c10::KernelFunction::call<at::Tensor&, at::Tensor&, at::Tensor const&, bool> (this=0x2bc45b8, opHandle=..., dispatchKeySet=..., args=false, args=false, args=false)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/boxing/KernelFunction_impl.h:67
#10 c10::Dispatcher::call<at::Tensor&, at::Tensor&, at::Tensor const&, bool>(c10::TypedOperatorHandle<at::Tensor& (at::Tensor&, at::Tensor const&, bool)> const&, at::Tensor&, at::Tensor const&, bool) const (
    this=0x7f1ad5307218 <c10::Dispatcher::realSingleton()::_singleton>, op=..., args=false, args=false, args=false) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/dispatch/Dispatcher.h:548
#11 c10::TypedOperatorHandle<at::Tensor& (at::Tensor&, at::Tensor const&, bool)>::call(at::Tensor&, at::Tensor const&, bool) const (this=0x7f1ad531df08 <at::_ops::copy_::call(at::Tensor&, at::Tensor const&, bool)::op>, args=false, args=false, args=false)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/dispatch/Dispatcher.h:414
#12 at::_ops::copy_::call (self=..., src=..., non_blocking=false) at aten/src/ATen/Operators_3.cpp:1855
#13 0x00007f1abb1f2f1c in at::Tensor::copy_ (this=0x7ffd781f3808, src=..., non_blocking=false) at aten/src/ATen/core/TensorBody.h:1987
#14 0x00007f1abb9bd843 in at::native::_to_copy (self=..., dtype=..., layout=..., device=..., pin_memory=..., non_blocking=false, optional_memory_format=...)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/native/TensorConversions.cpp:82
#15 0x00007f1abca4bc42 in at::(anonymous namespace)::(anonymous namespace)::wrapper___to_copy (self=..., dtype=..., layout=..., device=..., pin_memory=..., non_blocking=false, memory_format=...) at aten/src/ATen/RegisterCompositeExplicitAutograd.cpp:7944
#16 0x00007f1abcaf83b3 in c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>), &at::(anonymous namespace)::(anonymous namespace)::wrapper___to_copy>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > >::operator()(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) (this=0x32a4bd0, args=..., args=..., args=..., args=...,
    args=..., args=..., args=...) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/boxing/impl/WrapFunctionIntoFunctor.h:13
#17 c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>), &at::(anonymous namespace)::(anonymous namespace)::wrapper___to_copy>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > >, at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) (functor=0x32a4bd0, args=..., args=..., args=..., args=..., args=..., args=..., args=...)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/boxing/impl/make_boxed_from_unboxed_functor.h:424
#18 0x00007f1abbf38bc2 in c10::callUnboxedKernelFunction<at::Tensor, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > (
    unboxed_kernel_func=0x7f1abcaf80c0 <c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>), &at::(anonymous namespace)::(anonymous namespace)::wrapper___to_copy>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > >, at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)>, functor=0x32a4bd0, dispatchKeySet=..., args=...,
    args=..., args=..., args=..., args=..., args=..., args=...) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/boxing/KernelFunction_impl.h:57
#19 0x00007f1abbf39482 in c10::KernelFunction::call<at::Tensor, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > (this=0x2d4e4c8, opHandle=...,
    dispatchKeySet=..., args=..., args=..., args=..., args=..., args=..., args=..., args=...) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/boxing/KernelFunction_impl.h:67
#20 c10::Dispatcher::redispatch<at::Tensor, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> >(c10::TypedOperatorHandle<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)> const&, c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) const (this=0x7f1ad5307218 <c10::Dispatcher::realSingleton()::_singleton>, op=..., currentDispatchKeySet=..., args=..., args=..., args=..., args=..., args=..., args=..., args=...)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/dispatch/Dispatcher.h:557
#21 0x00007f1abbdbb2f5 in c10::TypedOperatorHandle<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)>::redispatch(c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) const (
    this=0x7f1ad530fec8 <at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)::op>,
    currentDispatchKeySet=..., args=..., args=..., args=..., args=..., args=..., args=..., args=...) at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/dispatch/Dispatcher.h:419
#22 at::_ops::_to_copy::redispatch (dispatchKeySet=..., self=..., dtype=..., layout=..., device=..., pin_memory=..., non_blocking=false, memory_format=...) at aten/src/ATen/Operators_0.cpp:6289
#23 0x00007f1abc76c9ff in at::(anonymous namespace)::_to_copy (self=..., dtype=..., layout=..., device=..., pin_memory=..., non_blocking=false, memory_format=...) at aten/src/ATen/RegisterBackendSelect.cpp:542
#24 0x00007f1abc76cf93 in c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>), &at::(anonymous namespace)::_to_copy>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > >::operator()(at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) (this=0x2dd6290, args=..., args=..., args=..., args=..., args=..., args=..., args=...)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/boxing/impl/WrapFunctionIntoFunctor.h:13
#25 c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>), &at::(anonymous namespace)::_to_copy>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat> > >, at::Tensor (at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, bool, c10::optional<c10::MemoryFormat>) (functor=0x2dd6290, args=..., args=..., args=..., args=..., args=..., args=..., args=...)
    at /home/ashari/build/debug_full/cmake-build/vcpkg-src/buildtrees/onnxruntime/pytorch.git/aten/src/ATen/core/boxing/impl/make_boxed_from_unboxed_functor.h:424

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 20.04
• ONNX Runtime installed from (source or binary): source
• ONNX Runtime version: commit cb31b7eab191bc6a0f9e051f136a18b709126614 (3/25/22)
• Python version: 3.8
• GCC/Compiler version (if compiling from source): gcc 10