tensorflow / tensorflow/tensorboard
Convertion of vectors to .bytes for standalone embedding projector
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 7.2k
- Forks
- 1.7k
- Avg merge
- 4d 22h
- Merged PRs (30d)
- 1
Description
Trying to use my own vectors/labels into the standalone embedding projector instead of the example model.
I did so as a comment below suggests and with following two files:
- vectors.bytes which is created with numpy's tofile() command:
array([[ 2.5136387 , -2.084486 , 0.23319365],
[ 4.008879 , 1.2343712 , -1.1920882 ],
[ 3.4046457 , -0.42918122, -0.84252584],
...,
[ 3.2287045 , -0.49425614, -0.7298379 ],
[ 0.9056233 , -0.4525969 , 2.068056 ],
[ 2.942347 , 0.94748515, -0.8100381 ]], dtype=float32)
vectors.tofile(os.path.join(LOG_DIR, "tensors.bytes")) (array size is 5000x3)
- meta.tsv which is a one column tab seperated file containing 5000 labels.
in standalone_projector_config.json I added
{
"embeddings": [
{
"tensorName": "Doc2Vec Lipid",
"tensorShape": [
5000,
3
],
"tensorPath": "/PATH/tensors.bytes",
"metadataPath": "/PATH/meta.tsv"
},
...
}
When uploading the vectors/meta files, in .TSV file format to embedding projector (either online or standalone), everything works as intended but including them in bytes/TSV format I get the following error message:
Bazel Closure Rules</h1><h3>//tensorboard/plugins/projector/vz_projector:standalone</h3><p>No srcs found in transitive closure with path component prefix matching request path.
@harveyslash The .bytes files are binary files filled with a bunch of 32 bit floating point numbers. You can use a numpy array with dtype=numpy.float32 and tofile to generate them:
vectors = numpy.zeros(vector_shape, dtype=numpy.float32)
vectors.tofile('my_tensors.bytes')
Here's a complete example of exporting spaCy Vocab vectors for use in the standalone projector.
https://github.com/explosion/spaCy/blob/master/examples/vectors_tensorboard_standalone.py
Originally posted by @justindujardin in https://github.com/tensorflow/tensorflow/issues/7562#issuecomment-376606027
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.