Ensemble model: Investigate optimal approach for incorporating covariates into model training

We may want to determine the best method for utilizing covariates in the training pipeline. Could either include these in the original data loading as an additional channel and update the weights for each batch, or we could utilize these in the end in the loss function. So far there are two potential approaches to evaluate:

**Approach 1: Include covariates as additional input channels**

- Add covariates to the input data during data loading
- Allow model to learn relationships between covariates and target through standard backpropagation
- Requires updating data loader and potentially model architecture

**Approach 2: Incorporate covariates in the loss function**

- Keep the current model architecture unchanged
- Use covariates to weight or adjust loss calculation after each training epoch
- Implement covariate-based loss modifications

Tasks:

- [ ] Implement Approach 1 (covariates as input channels)
- [ ] Implement Approach 2 (covariates in loss function)
- [ ] Run comparative experiments to evaluate both approaches
- [ ] Document performance metrics for each approach
- [ ] Determine optimal strategy based on results

Open questions:

- Which covariates are most predictive vs. nuisance variables?
- Could a hybrid approach (some covariates as inputs, others in loss) be beneficial?