[Performance][ARM64] ZVec Flat INT8 search is ~2-4x slower than FP32

## Benchmark Type

ARM64 Flat-search performance: INT8 versus FP32, with both unfiltered and scoped queries.

## Goal

When ZVec is used as the vector database backend of OpenViking, Flat search with INT8 quantization is significantly slower than unquantized FP32 on Apple Silicon ARM64.

Depending on the dataset size and query path:

- INT8 throughput is approximately **1.95x-4.01x lower**.
- INT8 average latency is approximately **1.95x-4.14x higher**.

Both configurations achieved 100% recall in this benchmark.

Our preliminary investigation suggests that the ARM64 implementation may not have an optimized INT8 distance-computation kernel and may fall back to scalar computation, while FP32 uses a NEON-optimized path. This is a hypothesis, not a confirmed root cause, and we would appreciate confirmation from the maintainers.

## Benchmark context

The results come from OpenViking's vector backend benchmark. It targets the `VikingVectorIndexBackend` boundary and uses the actual OpenViking context schema, tenant/URI scope construction, and ZVec adapter, while supplying precomputed query vectors.

Relevant code:

- [OpenViking backend benchmark](https://github.com/shirly121/OpenViking/blob/40506ba9657de0d3c2c0ff9a8848f67f3d3b638e/benchmark/vectordb_perf/run.py#L1177)
- [OpenViking ZVec adapter](https://github.com/shirly121/OpenViking/blob/40506ba9657de0d3c2c0ff9a8848f67f3d3b638e/openviking/storage/vectordb_adapters/zvec_adapter.py#L268)

Embedding generation, HTTP handling, LLM calls, and reranking are not included in the measured query latency.

The benchmark has three vector-search phases. This issue compares only:

1. `bare_vector_search`
2. `vector_search`, referred to below as **no directory filter**

The directory-filtered `filtered_vector_search` phase is intentionally excluded.

## Query path 1: bare vector search

This phase bypasses OpenViking's tenant-scope construction and calls the shared collection adapter directly:

```python
backend._shared_adapter.query(
query_vector=query.vector,
limit=top_k,
output_fields=RETRIEVAL_OUTPUT_FIELDS,
)
```

It does not pass `account_id`, `context_type`, `level`, visible URI roots, a directory URI prefix, or any additional DSL predicate.

The ZVec adapter translates it to an API call equivalent to:

```python
query = zvec.Query(
field_name="vector",
vector=query_vector,
param=None, # Flat index
)

docs = collection.query(
queries=query,
topk=top_k,
filter=None,
output_fields=[
"uri",
"level",
"context_type",
"abstract",
"active_count",
"updated_at",
"search_tags",
],
)
```

This is the closest measurement of raw ZVec Flat search in the OpenViking benchmark.

## Query path 2: no-directory-filter search

The `vector_search` phase calls the normal OpenViking backend interface:

```python
await backend.search_in_tenant(
ctx=benchmark_context,
query_vector=query.vector,
context_type="resource",
target_directories=None,
level=[2],
limit=top_k,
)
```

Although `target_directories=None`, this is not a filter-free query. OpenViking still constructs a base scope containing:

- `account_id = "bench_account"`
- `context_type = "resource"`
- `level IN [2]`
- the URI roots visible to the benchmark user

All synthetic records satisfy these predicates. Therefore, the filter still covers the entire 10K or 100K corpus, but is passed to and evaluated by ZVec.

The OpenViking filter AST is converted by the adapter into a ZVec filter string approximately equivalent to:

```text
account_id = 'bench_account'
AND context_type = 'resource'
AND level = 2
AND (
uri = '/resources'
OR uri LIKE '/resources/%'
OR uri = '/agent/skills'
OR uri LIKE '/agent/skills/%'
OR uri = '/user/bench_user'
OR uri LIKE '/user/bench_user/%'
)
```

Depending on OpenViking's account-bound backend wrapping, an equivalent `account_id` predicate may occur more than once. This does not change the filter semantics.

The final ZVec call is equivalent to:

```python
query = zvec.Query(
field_name="vector",
vector=query_vector,
param=None,
)

docs = collection.query(
queries=query,
topk=top_k,
filter=scope_filter_dsl,
output_fields=RETRIEVAL_OUTPUT_FIELDS,
)
```

| OpenViking phase | ZVec `filter` | Effective corpus |
| --- | --- | ---: |
| Bare search | `None` | Full corpus |
| No directory filter | Base tenant/context/level/URI scope | Full synthetic corpus |
| Directory-filtered search | Base scope plus directory URI prefix | Excluded from this issue |

## ZVec index configuration

Both variants use the same OpenViking schema. The vector field remains `DataType.VECTOR_FP32`, and FP32 query vectors are supplied in both cases. The index quantization setting is the relevant difference.

FP32:

```python
zvec.FlatIndexParam(
metric_type=zvec.MetricType.COSINE,
quantize_type=zvec.QuantizeType.UNDEFINED,
use_contiguous_memory=True,
)
```

INT8:

```python
zvec.FlatIndexParam(
metric_type=zvec.MetricType.COSINE,
quantize_type=zvec.QuantizeType.INT8,
use_contiguous_memory=True,
)
```

After ingestion, the collection is flushed, closed, and reopened with `read_only=True` and `enable_mmap=True` before warmup. This transition is outside the measured query phases.

## Methodology

Environment:

- macOS ARM64
- Apple M2 Pro, 12 CPU cores
- 32 GiB memory
- No GPU
- ZVec `0.2.2.dev282` for the recorded INT8 runs
- Flat index, cosine distance, 1024 dimensions
- Contiguous memory and read-only mmap snapshot
- INT8: `QuantizeType.INT8`
- FP32: `QuantizeType.UNDEFINED`

The latest ZVec release is currently `v0.7.0`. The numbers below are reported against the version actually used in the recorded runs rather than being presented as `v0.7.0` results. We can rerun the same workload on `v0.7.0` if this area has changed since `0.2.2.dev282`.

Workloads:

| Dataset | Queries | Concurrency | topK | Warmup |
| --- | ---: | ---: | ---: | ---: |
| 10K x 1024D | 100 | 4 | 20 | 10 |
| 100K x 1024D | 500 | 8 | 100 | 20 |

Warmup queries use the same concurrency and query path as the measured phase and are excluded from reported metrics.

## Baseline and results

### 10K x 1024D

| OpenViking query phase | ZVec Flat FP32 | ZVec Flat INT8 | Throughput regression | Average latency regression |
| --- | --- | --- | ---: | ---: |
| Bare search | 1703.578 QPS / 2.252 ms | 425 QPS / 9.32 ms | **4.01x lower QPS** | **4.14x higher latency** |
| No directory filter | 1048.603 QPS / 3.689 ms | 391 QPS / 10.15 ms | **2.68x lower QPS** | **2.75x higher latency** |

### 100K x 1024D

| OpenViking query phase | ZVec Flat FP32 | ZVec Flat INT8 | Throughput regression | Average latency regression |
| --- | --- | --- | ---: | ---: |
| Bare search | 137.807 QPS / 57.748 ms | 59 QPS / 133.58 ms | **2.34x lower QPS** | **2.31x higher latency** |
| No directory filter | 118.920 QPS / 66.731 ms | 61 QPS / 129.79 ms | **1.95x lower QPS** | **1.95x higher latency** |

Recall was 100% for both FP32 and INT8 in all cases.

The bare-search comparison is especially notable because both variants reach the same ZVec API with the same query vector, vector field, cosine metric, `topk`, `filter=None`, output fields, and contiguous read-only storage mode. The primary index-level difference is `QuantizeType.INT8` versus `QuantizeType.UNDEFINED`.

## Expected behavior

INT8 quantization is generally expected to reduce vector memory bandwidth and computation cost, or at least not cause a 2-4x search regression compared with FP32 Flat search under the same workload.

## Suspected cause and questions

Our current hypothesis is that ARM64 has a NEON-optimized FP32 distance path but no equivalent optimized INT8 kernel, causing INT8 to use a scalar fallback.

Could you please clarify:

1. Does ZVec currently provide an ARM64/NEON-optimized INT8 distance kernel for Flat search?
2. Is INT8 Flat search expected to fall back to scalar computation on Apple Silicon?
3. Are build options or runtime settings required to enable ARM64 INT8 optimization?
4. Has this path changed in `v0.7.0`, or is ARM64 INT8 optimization planned?
5. Is there a recommended profiling method for confirming which distance kernel is selected at runtime?

We can provide a standalone reproduction script or additional native profiling data if helpful.