Azure / Azure/data-api-builder
[Enh]: Embedding Overview
- Dominant language
- C#
- Stars
- 1.5k
- Forks
- 370
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 8
Description
### Phase 1
1. Provide the embedding subsystem
2. Cache embeddings in L1 & L2 cache
```json
{
"runtime": {
"embeddings": { } // new
}
}
```
- #3103
### Phase 2
1. Provide the `/embed` endpoint
2. Provide a chunking mechanism
```
https://{server}/embed
```
- #3103
### Phase 3
1. Introduce `embed` property to stored procedure parameters
2. Substitute incoming text with vector embeddings
```json
{
"entities": {
"{my-proc}": {
"source": {
"type": "stored-procedure",
"parameters": [
{
"auto-embed": true // new!
}
]
}
}
}
}
```
- #3331
### Phase 4
1. Introduce `semantic-search` properties to tables
2. Introduce `$semantic` keyword to REST
3. Introduce `$semantic` keyword to GraphQL
4. Search REDIS using vector search & return table records
```json
{
"entities": {
"{my-table}": {
"source": {
"type": "table"
},
"semantic-search": {} // new
}
}
}
```
- #3332
### Phase 5
1. Introduce `/embed/prewarm` endpoint
2. Support `$indexName`, `$table`, `$keyField`, `$textField`, `$batchSize` keywords
3. Batch embed and insert into Redis index
3. Return processed keys in response
```
https://{server}/embed/prewarm?
$indexName=redis-index-name&
$table=db-table-name&
$keyField=key-field-name&
$textField=text-field-name&
$batch=100&
$filter=(myFilters)
```
- #3333
Contributor guide
Assessment
This issue has not been assessed yet.