Embedding Aggregation of ICD-10 Labels
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Currently only text label embedding from billable code is supported. Explore and support if ingestion of ancestor label embeddings can improve proximity to noun chunks that represent an ICD-10 code. If so, persist aggregation method and add unit tests.
## Background
ICD-10 taxonomy is hyponym, ie, ancestor contain additional more general information. Simply averaging might coarsening too much the final embedding representing a billable code, but possibly a weighted approach is more successful.
Strategies to explore and to compare against “no aggregation":
1. Simple average
2. Weighted average: quadratic, exponential
3. Learnable weights (per depth, per code?)
## Motivation
Incorporating ancestor label description embeddings enriches the representation by including broader clinical context, such as the anatomical system or disease family, which may not be explicit in the most granular (billable) code’s description.
Weighting closer ancestors more than distant ones often captures specificity better. A natural weighting scheme is to use a decaying function such as quadratic, exponential, or learnable weights, so the direct label gets most influence, and broader ancestors contribute less signal.
## Draft for Learnable Weights
Let $C\in\mathbb{R}^{n\times 7\times d}$ be the tensor of code embeddings of dimension $d$ with first dimension listing the leaf node (=billable codes) and its ancestors from parent to root (exclusively). As nodes have depths in range of 3 to 7, codes with less than 6 ancestors are padded. We parameterize the weights $w$ and want to optimize the average distance/similarity between the weighted ancestor embeddings to text chunk examples (from annotated train set). As medical code distribution in real text has a long tail - most of them never occur in train sets or are very rare, we need to mitigate this through (a) generating samples, e.g. at least 5 (b) weighting by inverse of sample frequency per class: $\alpha_{c_i} = \frac{m_{c_i}}{m}$ to achieve equal importance weighting. Given a similarity function $sim$, the optimization problem formulated as a maximization problem is:
$w_{opt} = \arg\max_{w} (\frac{1}{n}\sum_{c_i}\alpha_{c_i}^{-1}\sum_{x^j_{c_i}} sim(c_iw_i, x_{c_i}^j))$.
The similiarity function could be cosine similarity, geodesic similarity, inverse Euclidean distance, angle.
Use minimizer (Adam, SGD, L-BFGS) to solve via gradient descent assuming sim/dist is differentiable.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.