tensorflow / tensorflow/java

Ability to convert Tensor to String representation

Open
#268 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
928
Forks
227
PR merge metrics
No merged PRs in 30d

Description

Per our discussion on Gitter, here is a possible implementation for converting Tensors to a String representation. It is still missing some important features, like collapsing long arrays using ellipses, but this can serve as a stepping stone. The functionality is meant to ease troubleshooting/debugging so performance should not be an issue.

import org.tensorflow.Session;
import org.tensorflow.ndarray.Shape;
import org.tensorflow.ndarray.buffer.DataBuffer;
import org.tensorflow.ndarray.buffer.DoubleDataBuffer;
import org.tensorflow.ndarray.buffer.FloatDataBuffer;
import org.tensorflow.ndarray.buffer.IntDataBuffer;
import org.tensorflow.ndarray.buffer.LongDataBuffer;
import org.tensorflow.ndarray.buffer.ShortDataBuffer;
import org.tensorflow.types.TFloat16;
import org.tensorflow.types.TFloat32;
import org.tensorflow.types.TFloat64;
import org.tensorflow.types.TInt32;
import org.tensorflow.types.TInt64;
import org.tensorflow.types.TUint8;

import java.util.StringJoiner;

public final class Tensors
{
	private final Session session;

	/**
	 * @param session the session used by all operations
	 */
	public Tensors(Session session)
	{
		this.session = session;
	}

	/**
	 * @param tensor a tensor
	 * @return the String representation of the tensor
	 */
	public String toString(TFloat64 tensor)
	{
		Shape shape = tensor.shape();
		DoubleDataBuffer doubles = tensor.asRawTensor().data().asDoubles();
		return toString(doubles, shape, 0, 0, tensor.rank()).text;
	}

	/**
	 * @param tensor a tensor
	 * @return the String representation of the tensor
	 */
	public String toString(TFloat32 tensor)
	{
		Shape shape = tensor.shape();
		FloatDataBuffer doubles = tensor.asRawTensor().data().asFloats();
		return toString(doubles, shape, 0, 0, tensor.rank()).text;
	}

	/**
	 * @param tensor a tensor
	 * @return the String representation of the tensor
	 */
	public String toString(TFloat16 tensor)
	{
		Shape shape = tensor.shape();
		FloatDataBuffer doubles = tensor.asRawTensor().data().asFloats();
		return toString(doubles, shape, 0, 0, tensor.rank()).text;
	}

	/**
	 * @param tensor a tensor
	 * @return the String representation of the tensor
	 */
	public String toString(TInt64 tensor)
	{
		Shape shape = tensor.shape();
		LongDataBuffer doubles = tensor.asRawTensor().data().asLongs();
		return toString(doubles, shape, 0, 0, tensor.rank()).text;
	}

	/**
	 * @param tensor a tensor
	 * @return the String representation of the tensor
	 */
	public String toString(TInt32 tensor)
	{
		Shape shape = tensor.shape();
		IntDataBuffer doubles = tensor.asRawTensor().data().asInts();
		return toString(doubles, shape, 0, 0, tensor.rank()).text;
	}

	/**
	 * @param tensor a tensor
	 * @return the String representation of the tensor
	 */
	public String toString(TUint8 tensor)
	{
		Shape shape = tensor.shape();
		ShortDataBuffer doubles = tensor.asRawTensor().data().asShorts();
		return toString(doubles, shape, 0, 0, tensor.rank()).text;
	}

	/**
	 * @param data      the data
	 * @param shape     the shape of the tensor
	 * @param index     the index of the tensor element to start at
	 * @param dimension the current dimension
	 * @param rank      the maximum dimension
	 * @return the String representation of the {@code dimension}
	 */
	private ToStringResponse toString(DataBuffer<?> data, Shape shape, int index, int dimension, int rank)
	{
		int numElements = 0;
		StringJoiner joiner;
		if (dimension < rank)
		{
			joiner = new StringJoiner(",\n", "\t".repeat(dimension) + "[\n", "\n" + "\t".repeat(dimension) + "]");
			for (long i = 0, size = shape.size(rank - 1); i < size; ++i)
			{
				ToStringResponse response = toString(data, shape, index, dimension + 1, rank);
				joiner.add(response.text);
				numElements += response.numElements;
				index += response.numElements;
			}
		}
		else
		{
			joiner = new StringJoiner(",", "\t".repeat(dimension) + "[", "]");
			for (long i = 0, size = shape.size(rank - 1); i < size; ++i)
			{
				joiner.add(String.valueOf(data.getObject(index)));
				++numElements;
				++index;
			}
		}
		return new ToStringResponse(joiner.toString(), numElements);
	}

	/**
	 * @param text        the string representation of a tensor dimension
	 * @param numElements the number of elements contained in {@code text}
	 */
	private record ToStringResponse(String text, int numElements)
	{
	}
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue contains a proposed Tensors implementation using Session, tensor type classes, Shape, and DataBuffer APIs, but names no repository file or test. Start by locating the existing tensor and string-conversion entry points, then review the proposal and its missing long-array collapsing behavior. Done means the supported tensor types have an agreed, tested string representation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.