Tensor Scope and resource management

cc @saudet @karllessard @Craigacp

This issue is for TensorScope and tensor resource management more generally, as discussed in #181 and the community call.

I'm envisioning usage like (or with try-with-resources):

TensorScope scope = new TensorScope();
TInt32 input = stuff;
TInt32 result = function(input).detach();
scope.close();
// result is accessible here, input and any tensors created in function() are not

A few issues I'd like comment on:

• NDArrays. As mentioned in https://github.com/tensorflow/java/issues/181#issuecomment-755863642, it's possible to have a NDArray opaquely backed by a tensor. The tensor could be closed by a TensorScope, making the NDArray inaccessible in a way that probably won't make sense to users. I plan to add isNativeBuffer() and closeNativeBuffer() to NDArray, and some Javadoc comments about this, so I think it's ok as the default behavior, but I also think it would be a good idea to have TensorScope have an option to copy out NDArrays on close (i.e. to a Java buffer). Not sure how it would be implemented yet, but it should be possible. When exactly to do it is more complicated. We don't want to do it for every NDArray, because that would include every TType, but we may want to do it for non-TType NDArrays that use one of those buffers.
• Threading: how much do we want to support multithreading? PointerScope uses ThreadLocal, which is necessary for the global scope stacks, but prevents running parts of a model in another thread, if that's even supported in the first place.
• PointerScope (@saudet). We discussed implementing this by wrapping PointerScope, but that means that as far as I understand it, the PointerScope would pick up any other pointers, too. It seems better to re-implement the tracking ourselves, which would be necessary for things like copying out NDArrays anyways, and using TF_Tensor's reference counting. RawTensor also already uses PointerScope internally, so I think that takes care of the reference counting.
• TF_Tensor's deallocator doesn't implement ReferenceCounter, which as far as I can tell will make the reference counting not work. @saudet
• TF_Tensor and TFE_TensorHandle. Do I need to track TFE_TensorHandle as well?
• More broadly, this waits until the end of the scope to do any cleanup, where especially for eager mode we want to remove temporary variables as soon as they are un-live. Am I correct that when using Operands in eager mode, we don't actually realize the tensors in Java and cleanup is done by TF's native side?

TensorMapper#nativeHandle also probably needs to call retainReference, depending on the semantics we want.

Commencez par lire l’issue #181 et la discussion autour de TensorScope, puis examinez PointerScope, RawTensor, NDArray et TensorMapper#nativeHandle. Vérifiez comment fonctionne actuellement le comptage des références de TF_Tensor et TFE_TensorHandle ; le travail ne sera terminé que lorsque les sémantiques de scope, de threading, de copie et de cleanup auront été décidées et implémentées.