Define and Build Vector Embedding Module
- Linguagem predominante
- Python
- Estrelas
- 1
- Forks
- 1
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
Input: retrieves dataframe with columns as named in chunking strategy module and LLM access/store access (precomputed!)
Output: a dataframe with the index (report id, sentence id, chunk id) of size n x 4, the chunk text and a tensor with the embeddings of size n x d where d is embedding dimensionality
Behaviour: stores newly computed embeddings locally in a fixed directory (config) or if already calculated loads embeddings from the fixed directory, or completes calculation if only precomputed partially. In case a dimensionality reduction is requested as indicated by experimental config, request is delegated to child class.
In the child class, embeddings are either reduced and stored in a separate location or if already calculated the reduced embeddings are loaded directly bypassing the original embeddings. If only available in parts, requests for original embeddings are posed to parent class. Hence the vectore embedding module needs to provide a method that returns embeddings given a multi-index.
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.