Tensor Scope and resource management

cc @saudet @karllessard @Craigacp

This issue is for TensorScope and tensor resource management more generally, as discussed in #181 and the community call.

I'm envisioning usage like (or with try-with-resources):

TensorScope scope = new TensorScope();
TInt32 input = stuff;
TInt32 result = function(input).detach();
scope.close();
// result is accessible here, input and any tensors created in function() are not

A few issues I'd like comment on:

• NDArrays. As mentioned in https://github.com/tensorflow/java/issues/181#issuecomment-755863642, it's possible to have a NDArray opaquely backed by a tensor. The tensor could be closed by a TensorScope, making the NDArray inaccessible in a way that probably won't make sense to users. I plan to add isNativeBuffer() and closeNativeBuffer() to NDArray, and some Javadoc comments about this, so I think it's ok as the default behavior, but I also think it would be a good idea to have TensorScope have an option to copy out NDArrays on close (i.e. to a Java buffer). Not sure how it would be implemented yet, but it should be possible. When exactly to do it is more complicated. We don't want to do it for every NDArray, because that would include every TType, but we may want to do it for non-TType NDArrays that use one of those buffers.
• Threading: how much do we want to support multithreading? PointerScope uses ThreadLocal, which is necessary for the global scope stacks, but prevents running parts of a model in another thread, if that's even supported in the first place.
• PointerScope (@saudet). We discussed implementing this by wrapping PointerScope, but that means that as far as I understand it, the PointerScope would pick up any other pointers, too. It seems better to re-implement the tracking ourselves, which would be necessary for things like copying out NDArrays anyways, and using TF_Tensor's reference counting. RawTensor also already uses PointerScope internally, so I think that takes care of the reference counting.
• TF_Tensor's deallocator doesn't implement ReferenceCounter, which as far as I can tell will make the reference counting not work. @saudet
• TF_Tensor and TFE_TensorHandle. Do I need to track TFE_TensorHandle as well?
• More broadly, this waits until the end of the scope to do any cleanup, where especially for eager mode we want to remove temporary variables as soon as they are un-live. Am I correct that when using Operands in eager mode, we don't actually realize the tensors in Java and cleanup is done by TF's native side?

TensorMapper#nativeHandle also probably needs to call retainReference, depending on the semantics we want.

Lesen Sie zunächst Issue #181 und die Diskussion rund um TensorScope, und untersuchen Sie anschließend PointerScope, RawTensor, NDArray und TensorMapper#nativeHandle. Prüfen Sie, wie die Referenzzählung von TF_Tensor und TFE_TensorHandle derzeit funktioniert; die Arbeit ist erst abgeschlossen, wenn die Semantik von Scope, Threading, Kopieren und Cleanup entschieden und implementiert ist.