dwyl / dwyl/learn-postgresql

PGVECTOR-ELIXIR extension: example with the classification of data using OpenAI

Open
#103 3 comments 1 reaction 0 assignees View on GitHub
external-dependency
Dominant language
JavaScript
Stars
346
Forks
37
PR merge metrics
No merged PRs in 30d

Description

# 1. An example of a usage of `pgvector`.

We want to classify a dataset using "a certain distance" among the rows. The idea is to send the data to the (rate limited) [OpenAI api endpoint](https://platform.openai.com/docs/api-reference/introduction), get in return tokens for each row, store as a vector, and then apply a distance sorting query between rows based on this vector field.

Note: the distance operator `<=>` is also used with `PostGIS`.

[Screenshot 2023-05-09 at 10 09 35](https://www.crunchydata.com/blog/whats-postgres-got-to-do-with-ai)

# 2. `pgvector` and indexes

[Screenshot 2023-05-09 at 16 48 03](https://www.crunchydata.com/blog/pgvector-performance-for-developers)

A summary of the post above, as found in the post!:
`pgvector `provides a comprehensive, performant, and 100% open source database for vector data. As a developer, the key to getting performance from `pgvector` are:

- Ensure your query is using the indexes
- Ensure your indexes have the optimal list size
- Ensure you have enough memory for the index

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.