django / django/new-features

Add MariaDB 11.7+ vector field and vector distance expressions

Open
#151 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
188
Forks
7
PR merge metrics
No merged PRs in 30d

Description

### Code of Conduct

- [x] I agree to follow Django's Code of Conduct

### Feature Description

MariaDB 11.7+ provides native vector support through the VECTOR column type and built-in functions such as VEC_FromText() and vector distance functions. Django currently has no native ORM-level support for storing vector values in MariaDB models or querying them through expressions and annotations.

I’ve built a third-party library around this functionality already, which confirms the feature is practical and usable today. However, native Django support would be significantly more beneficial because it would allow MariaDB vector functionality to integrate cleanly with Django’s field system, schema generation, migrations, introspection, and ORM expressions without requiring each project to maintain custom glue code.

This proposal is specifically about MariaDB support within Django’s existing MariaDB backend, not a generic cross-database vector abstraction.

### Problem

## Developers using Django with MariaDB cannot currently model vector columns or query vector distances using built-in Django primitives.

### Today, common use cases require:

- custom model fields
- raw SQL
- custom expressions/functions
- manual schema handling
- extra third-party code for database integration

### This creates friction for otherwise standard Django applications that want to:

- store embeddings on models
- annotate querysets with vector distance
- order results by similarity
- serialize and deserialize vector values reliably
- manage schema for vector columns in a Django-native way

Because MariaDB already supports these capabilities at the database level, the missing piece is first-class Django integration.

### Request or proposal

proposal

### Additional Details

## Why this is useful:
- Vector embeddings are becoming a normal part of application development, not just specialised ML systems.
- Many Django apps using MariaDB would benefit from storing embeddings directly in their relational models.
- Native support would reduce reliance on raw SQL and make this functionality more discoverable, testable, and maintainable.
- It would also make Django’s MariaDB backend feel more complete relative to the database features MariaDB already exposes.

## Why not only third-party:

- I already built a library around this functionality, which shows it can be implemented from user space.
- But native support would be much more valuable because the feature touches core ORM/database integration points such as field definition, migrations, schema generation, introspection, and expression support.
- A built-in implementation would give users a more consistent and reliable API and avoid fragmentation across separate packages.

## Scope expectations:

- MariaDB-only
- version-gated to MariaDB 11.7+
- not proposing a universal vector API across all databases in the first step

### Implementation Suggestions

## A reasonable initial scope could be:

- Add a MariaDB-compatible VectorField(dim=...) for model definitions.
- Support schema generation so the field maps to VECTOR().
- Add value preparation and conversion so Python iterables can round-trip cleanly to and from MariaDB vector values.
- Add ORM expressions/functions for:
- VEC_FromText()
- VEC_DISTANCE()
- VEC_DISTANCE_COSINE()
- VEC_DISTANCE_EUCLIDEAN()
- Add backend/version checks so the feature is only available on supported MariaDB versions.
- Add database introspection support so existing vector columns can be recognized correctly.

## I would suggest keeping the first version intentionally small:

- field support
- serialization/deserialization
- distance/query expressions
- backend gating
- tests and documentation

I would not require vector index support in the first version unless maintainers feel it belongs in the initial scope, since index APIs may need more design discussion than field/expression support.

## Possible Example API
```
from django.db import models

class Document(models.Model):
title = models.CharField(max_length=255)
embedding = models.VectorField(dim=768)
```

```
from django.db.models import F
from django.db.models.functions import VecDistanceCosine

results = (
Document.objects
.annotate(distance=VecDistanceCosine(F("embedding"), query_vector))
.order_by("distance")
)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.