aieoa / aieoa/icd10-lab

Define and Build Vector Embedding Module

Aberta
#15 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
1
Forks
1
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

Input: retrieves dataframe with columns as named in chunking strategy module and LLM access/store access (precomputed!)
Output: a dataframe with the index (report id, sentence id, chunk id) of size n x 4, the chunk text and a tensor with the embeddings of size n x d where d is embedding dimensionality

Behaviour: stores newly computed embeddings locally in a fixed directory (config) or if already calculated loads embeddings from the fixed directory, or completes calculation if only precomputed partially. In case a dimensionality reduction is requested as indicated by experimental config, request is delegated to child class.
In the child class, embeddings are either reduced and stored in a separate location or if already calculated the reduced embeddings are loaded directly bypassing the original embeddings. If only available in parts, requests for original embeddings are posed to parent class. Hence the vectore embedding module needs to provide a method that returns embeddings given a multi-index.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.