PyTorch Lightning and better T-S distillation

I'm trying to model the following problem with neural networks using knowledge distillation. I will lay out the setting. Your goal is to suggest the best possible options I could opt for. I am building a framework around all of that, and would love users to have all the best options at their disposal to choose from.

So, we have two NNs, a teacher and a student. Roughly speaking, the two models take images as input as produce a single point estimate score. The teacher is not only larger, but also accept one extra modality than the student. The distillation goal would be to train an equally accurate model while dropping that extra modality in the student. The models are for image quality assessment (IQA). Precisely, for panorama IQA. To dive deeper into the inputs format, the teacher model receives records made of the large high-resolution panorama image, and the corresponding set of lower resolution constituent images. A large backbone processes the panorama image into a panorama feature map, while a smaller backbone processes each constituent image, pooling their representation together into a single constituent feature map. The diff feature map between the two is the overall network feature map passed to the model head. On the contrary, the student model receives records made only of the panorama image, only has the panorama backbone, and the overall network feature map coincides with the panorama feature map. The options for the head are basically identically in both cases, starting from the overall feature map. The end user can choose between two different types of heads: a point estimate head, or a prediction intervals head (PIVEN). The point estimate head receives the overall feature map and produces a single floating point quality score, between 0 and 1 (or 0 and 100, whatever scale you think gives the model more stability during training). On the other end, the PIVEN head receives the overall feature map, and produces a feature vector of size three, where the first two entries map the upper and lower bound of the prediction interval, and the third and last entry is a normalized weight (passed through the sigmoid function) between 0 and 1 that polarizes where the value is within the prediction interval. The point estimate from the PIVEN head can be recovered with a specific formula that uses the normalized weight to identify the value estimate within the prediction interval, with the added bonus that the size of the interval can be used as a noise, and therefore confidence, estimate. The point estimates are evaluated against the target quality score using a standard MSE loss (or an equivalent regression loss).
The framework should allow end users to create any of the six combinations of models (teacher + point head, teacher + PIVEN head, student + point head, student + PIVEN head, student + point head via distillation, student + PIVEN head via distillation), with any regression loss between point estimates and target quality scores. You have to tell me how would you structure knowledge distillation, where would you tie it, and if there are multiple valuable options for it that we should offer to our end users. Remember, I don't need neither generic distillation options, nor all possibile knowledge distillation options for the case I just described, rather only the valuable ones that are achievable and would make sense to offer.