Slow iteration/converting of tensors to vector<float> via Python C API

System information

• Have I written custom code: Yes
• OS Platform and Distribution: Windows 10 Build 19041 and Docker version 19.03.12
• TensorFlow installed from: binary
• TensorFlow version: v2.17.0-rc1-2-gad6d8cc177d 2.17.0
• Python version: 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]
• CUDA/cuDNN version: No GPU
• GPU model and memory: No GPU

Describe the current behavior

Currently iteration/converting of tensor to vector<float> via Python C API is extremely slow - it takes from 13 seconds to a minute to convert 100 tensors (with shape (1792,)) into c++ vector<float>.

If the tensor is converted into ndarray with numpy() call beforehand the operation is performed instantaneously.

Describe the expected behavior

The speed of tensor conversion and the speed of ndarray conversion is the same (or similar).

Standalone code to reproduce the issue

Collab Notebook

from annoy import AnnoyIndex
import tensorflow as tf
from time import perf_counter

tf.compat.v1.enable_eager_execution()

dims = 1792
trees = 10000
features = []

for key in range(0, 100):
    features.append(tf.random.uniform([dims]))

t1 = perf_counter()

t = AnnoyIndex(dims, metric='angular')

for key, feature in enumerate(features):
    # t.add_item(key, feature.numpy())
    t.add_item(key, feature)

t2 = perf_counter()

print(f"Vector add: {t2 - t1:.2f}")

Other info / logs

The issue is present in both Tensorflow 1 and 2. Tested in Docker.

• Tensorflow: 2.17.0 (tensorflow/tensorflow:latest-jupyter): Vector add: 52.62
• Tensorflow: 2.17.0 with numpy() call (tensorflow/tensorflow:latest-jupyter): Vector add: 0.03
• Tensorflow: 2.3.0 (tensorflow/tensorflow:latest-jupyter): Vector add: 28.09
• Tensorflow: 2.3.0 with numpy() call (tensorflow/tensorflow:latest-jupyter): Vector add: 0.02
• Tensorflow: 1.15.2 (tensorflow/tensorflow:1.15.2-py3-jupyter): Vector add: 29.82

Same issue in annoy library.
Maybe relevant Tensorflow issue. I tested on Numpy 1.19.1 and it did not help.

Annoy library call hierarchy:

• add_item
• py_an_add_item
• convert_list_to_vector
  • int z - cycle iterator
  • int f - tensor length
  • PyObject* v - tensor
  • PyObject *pf - one tensor value

  for (int z = 0; z < f; z++) {
    PyObject *key = PyInt_FromLong(z);
    PyObject *pf = PyObject_GetItem(v, key);
    (*w)[z] = PyFloat_AsDouble(pf);
    Py_DECREF(key);
    Py_DECREF(pf);
  }