aieoa / aieoa/icd10-lab

Embedding Aggregation of ICD-10 Labels

Aberta
#20 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
1
Forks
1
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

Currently only text label embedding from billable code is supported. Explore and support if ingestion of ancestor label embeddings can improve proximity to noun chunks that represent an ICD-10 code. If so, persist aggregation method and add unit tests.

## Background
ICD-10 taxonomy is hyponym, ie, ancestor contain additional more general information. Simply averaging might coarsening too much the final embedding representing a billable code, but possibly a weighted approach is more successful.
Strategies to explore and to compare against “no aggregation":

1. Simple average
2. Weighted average: quadratic, exponential
3. Learnable weights (per depth, per code?)

## Motivation
Incorporating ancestor label description embeddings enriches the representation by including broader clinical context, such as the anatomical system or disease family, which may not be explicit in the most granular (billable) code’s description.
Weighting closer ancestors more than distant ones often captures specificity better. A natural weighting scheme is to use a decaying function such as quadratic, exponential, or learnable weights, so the direct label gets most influence, and broader ancestors contribute less signal.

## Draft for Learnable Weights
Let $C\in\mathbb{R}^{n\times 7\times d}$ be the tensor of code embeddings of dimension $d$ with first dimension listing the leaf node (=billable codes) and its ancestors from parent to root (exclusively). As nodes have depths in range of 3 to 7, codes with less than 6 ancestors are padded. We parameterize the weights $w$ and want to optimize the average distance/similarity between the weighted ancestor embeddings to text chunk examples (from annotated train set). As medical code distribution in real text has a long tail - most of them never occur in train sets or are very rare, we need to mitigate this through (a) generating samples, e.g. at least 5 (b) weighting by inverse of sample frequency per class: $\alpha_{c_i} = \frac{m_{c_i}}{m}$ to achieve equal importance weighting. Given a similarity function $sim$, the optimization problem formulated as a maximization problem is:
$w_{opt} = \arg\max_{w} (\frac{1}{n}\sum_{c_i}\alpha_{c_i}^{-1}\sum_{x^j_{c_i}} sim(c_iw_i, x_{c_i}^j))$.

The similiarity function could be cosine similarity, geodesic similarity, inverse Euclidean distance, angle.
Use minimizer (Adam, SGD, L-BFGS) to solve via gradient descent assuming sim/dist is differentiable.

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.