AOSSIE-Org / AOSSIE-Org/Ell-ena

feat: meeting transcription from Vexa API is stored but not chunked before embedding — transcriptions longer than the embedding model's context window are silently truncated, losing the end of long meetings

Open
#290 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Dart
Stars
54
Forks
110
PR merge metrics
No merged PRs in 30d

Description

## 🐛 Problem Statement

Ell-ena's RAG pipeline (documented in `supabase/functions/generate-embeddings/`) generates vector embeddings from meeting transcriptions for semantic search. The Gemini embedding API has a **maximum input token limit** (2,048 tokens for `text-embedding-004`, or 3,072 for `embedding-001`). A meeting transcription longer than this limit is silently truncated at the embedding layer.

For a 2-hour engineering meeting (which easily produces 5,000-10,000 token transcriptions), this means the embedding only captures the first ~30 minutes. Searching for *"what did we decide about the API design in the second hour?"* returns no results — the second hour's content was never embedded.

The README documents: *"RAG (Retrieval-Augmented Generation) implementation for contextual intelligence"* — but RAG silently failing for long meetings directly contradicts this claim.

## Root Cause

`supabase/functions/generate-embeddings/index.ts` almost certainly sends the full transcription text as a single string to the embedding API without splitting it into overlapping chunks first.

## Proposed Fix

Implement a sliding window chunker before embedding generation:

```typescript
// supabase/functions/generate-embeddings/index.ts

const CHUNK_SIZE_TOKENS = 512; // Safe chunk size (well under limits)
const CHUNK_OVERLAP_TOKENS = 64; // Overlap ensures context continuity across chunks

function chunkTranscription(text: string): string[] {
// Approximate tokenization: ~4 chars per token for English
const charsPerChunk = CHUNK_SIZE_TOKENS * 4;
const overlapChars = CHUNK_OVERLAP_TOKENS * 4;
const chunks: string[] = [];

// Split on sentence boundaries to avoid cutting mid-sentence
const sentences = text.split(/(?<=[.!?])\s+/);
let currentChunk = '';

for (const sentence of sentences) {
if ((currentChunk + sentence).length > charsPerChunk) {
chunks.push(currentChunk.trim());
// Keep last `overlapChars` for continuity
currentChunk = currentChunk.slice(-overlapChars) + sentence;
} else {
currentChunk += ' ' + sentence;
}
}
if (currentChunk.trim()) chunks.push(currentChunk.trim());
return chunks;
}

// In the embedding function:
const chunks = chunkTranscription(transcriptionText);
const embeddings = await Promise.all(
chunks.map(chunk => generateEmbedding(chunk))
);

// Store each chunk as a separate embedding with metadata:
// { meeting_id, chunk_index, chunk_text, embedding, created_at }
```

Update the `meeting_vector_search` Supabase function (`sqls/09_meeting_vector_search.sql`) to return the chunk-level result but group by `meeting_id` for the final response.

## Files to Modify

| File | Change |
|---|---|
| `supabase/functions/generate-embeddings/index.ts` | Add sliding window chunker before embedding call |
| `sqls/09_meeting_vector_search.sql` | Update schema to store `chunk_index` alongside embeddings |
| `supabase/migrations/` | Add migration for `chunk_index` column |

**Suggested labels:** `bug`, `rag`, `backend`, `level: intermediate`

I would like to work on this. Could you please assign it to me?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.