The generated embedding contains accuracy errors.

Nobody has claimed this yet.

### Feature request

I want to use gte-qwen2-7B to generate embedding, but after trying infinity I found the result is not reliable.

Results from sentencetransformer
Results from infinity with 1 sentence and 1 engine

Results from infinity with multiple same sentence and 4 engine

## Reproduction

Code for transformers:

```python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("/mnt/dolphinfs/hdd_pool/docker/user/hadoop-aipnlp/INS/ruanjunhao04/models/gte-Qwen2-7B-instruct/main",model_kwargs={'torch_dtype':'auto'}, trust_remote_code=True,).cuda()
# In case you want to reduce the maximum length:
model.max_seq_length = 8192

documents = [
"Paris is in France."
]

document_embeddings = model.encode(documents,precision='float32',convert_to_numpy=True)
print(document_embeddings)

```

Code for inifinity:
```python

import asyncio
from infinity_emb import AsyncEngineArray, EngineArgs, AsyncEmbeddingEngine
from torch.cuda import device_count
import numpy as np

# List of sentences to embed
sentences = ["Paris is in France."]

# Number of available GPUs (engines)
engine_count =1
# device_count()
# Split sentences into roughly equal chunks for each engine
def split_sentences(sentences, engine_count):
# Split sentences into `engine_count` chunks
avg_len = len(sentences) // engine_count
chunks = [sentences[i * avg_len: (i + 1) * avg_len] for i in range(engine_count)]
# If there are leftovers, add them to the last chunk
if len(sentences) % engine_count != 0:
chunks[-1].extend(sentences[engine_count * avg_len:])
return chunks

# Set up multiple engines (one for each GPU)
array = AsyncEngineArray.from_args([
EngineArgs(
model_name_or_path="/mnt/dolphinfs/hdd_pool/docker/user/hadoop-aipnlp/INS/ruanjunhao04/models/gte-Qwen2-7B-instruct/main",
engine="torch",
embedding_dtype="float32",
dtype="auto",
device='cuda',
device_id=f'{i}',
served_model_name=f'model_{i}'
) for i in range(engine_count)
])

async def embed_text(engine: AsyncEmbeddingEngine, sentences_chunk: list):
async with engine:
embeddings, _ = await engine.embed(sentences=sentences_chunk)
return embeddings

async def run_parallel_embeddings():
# Split the sentences into chunks for each engine
sentence_chunks = split_sentences(sentences, engine_count)

tasks = []
# Create a task for each engine, passing the respective chunk of sentences
for i in range(engine_count):

tasks.append(embed_text(array[i], sentence_chunks[i]))

# Wait for all embeddings to complete
results = await asyncio.gather(*tasks)

combined_embeddings = np.concatenate([result for result in results], axis=0)

return combined_embeddings

# Run the async function to embed the text in parallel
results = asyncio.run(run_parallel_embeddings())

# Optionally, handle or display the embeddings here
# For example:
print(f"Results: {results}")

```

### Motivation

A bug that need to be solved.

### Your contribution

.

Start by running the provided SentenceTransformer and infinity snippets with gte-qwen2-7B, then compare the resulting embeddings under the same inputs and settings. Trace the AsyncEmbeddingEngine.embed path and torch engine configuration; done should include a reproducible explanation of the discrepancy and a validated correction or clearly documented expected behavior.

Written by the indexing model from the issue text.