django / django/new-features

Add Native Vector Support for the Oracle: VectorField, VectorIndex, and VectorDistance

Open
#60 14 comments 7 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
188
Forks
7
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [x] I agree to follow Django's Code of Conduct

### Feature Description

We propose official support in Django for the Oracle Database’s native VECTOR data type, introduced in the Oracle Database 23.4. This includes:

### Features Included:
**VectorField model field:**

- Accepts optional dimensions, storage_format, and storage_type arguments.
- Supports Dense and Sparse vector storage.
- Auto-converts lists, arrays, and oracledb.SparseVector for insert/update.

**Vector Index support:**

- VectorIndex class using Meta.indexes.
- Support for HNSW and IVF index types.
- Optional parameters: distance, accuracy, parallel, etc.

**Vector distance expressions and lookups:**

- Custom Func class VectorDistance for VECTOR_DISTANCE(lhs, rhs, metric)
- CosineDistance, EuclideanDistance, and NegativeDotProduct etc. as lookups.
- Query syntax via filter() and order_by() for similarity search.

**Testing**

- Dense and Sparse vector insert/query tests added.
- Stress test scripts for repeated inserts/queries included.

This brings Django ORM in line with modern AI/ML and search workloads using vector embeddings (e.g., images, text, semantic search).

### Problem

Django currently does not natively support the Oracle Database's VECTOR data type. This limits users of the Oracle 23ai who want to:
-Store and query vector embeddings directly in the database.
-Perform similarity search using the Oracle's VECTOR_DISTANCE() function.
-Leverage the Oracle's native VECTOR indexing (e.g., IVF, HNSW) for high-performance nearest neighbor search.

Without first-class Django support, developers must fall back to raw SQL or manually patch fields and expressions, leading to poor maintainability and loss of ORM benefits.

### Request or proposal

proposal

### Additional Details

### Implementation Status
**We have already implemented:**

- Custom VectorField with support for DENSE and SPARSE formats
- Automatic SQL generation for model/table creation
- VectorIndex support with customizable parameters and distance metrics
- ORM expressions and lookups for vector distance queries (e.g., CosineDistance, EuclideanDistance)
- Basic tests for dense vector creation, insertion, indexing, and querying
- Integration with Oracle’s Python driver (oracledb) for runtime behavior

**Example:**
```
from django.db import models
VectorIndex = model.VectorIndex
VectorDistanceType = models.VectorDistanceType
VectorIndexType = models.VectorIndexType

class Product(models.Model):
name = models.CharField(max_length=100)
embedding = models.VectorField(dim=3, storage_format=VectorStorageFormat.FLOAT32, storage_type=VectorStorageType.DENSE)

class Meta:
indexes = [
VectorIndex(
fields=["embedding"],
name="vec_idx_product",
index_type=VectorIndexType.HNSW,
distance=VectorDistanceType.COSINE,
)
]

```

And a Similarity search can be performed

```
query_vector = array.array("f", [1.0, 2.0, 3.0])

products = Product.objects.annotate(
score=VectorDistance(
"embedding",
query_vector,
metric=VectorDistanceType.COSINE,
)
).order_by("score")[:5]
```

### Implementation Suggestions

_No response_

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.