deepskies / deepskies/DeepUQ

Possible Error in DER Loss, Matching Losses from TF Implementation

Open
#205 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Dear all,
I came across the ML4PS paper regarding this work recently, and found it super useful to a study I am conducting! Great work.

I was wondering if someone could look into the definition of the DER loss. I am comparing with this TensorFlow implementation from the original paper [here](https://github.com/fbohu/evidential-quantile-regression/blob/177b4a5402a6eb2cc6aec95c940d52910a928311/quantilelosses.py#L49) . Firstly, in line 453 in your models.py, I think there should be a 1/beta term. You have
```python
+ (coeff * torch.abs(error / w_st) * (2.0 * nu + alpha))
```
but the [original paper](https://arxiv.org/pdf/2308.10650) defines in equation 12 this part to be $\rho_q (y_i − \gamma_i) · \phi$ where $\phi_i = 2 \nu_i + \alpha_i + 1/\beta_i$

Additionally, they define the width of the distribution through sampling an exponential as such

```python
tfd = tfp.distributions
theta = (1.0-2.0*quantile)/(quantile*(1.0-quantile))
mean_ = beta/(alpha-1)

w_i_dis = tfd.Exponential(rate=1/mean_)
```
However, it appears you guys define it via

```python
w_st = torch.sqrt(beta * (1 + nu) / (alpha * nu))
```

I'm a little new to this area, so it's very possible I'm missing something. I'm also happy to submit a PR and contribute to the development of this package.

All the best,
Eddie

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.