facebook / facebook/prophet

Decouple performance measures

Open
#2,234 0 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue
Dominant language
Python
Stars
20.4k
Forks
4.6k
Avg merge
19h 52m
Merged PRs (30d)
1

Description

Currently, `diagnostics.performance_metrics()` contains a function scoped list of available metrics: https://github.com/facebook/prophet/blob/95b1741d447c7283363475a63f1ceceaec2981c9/python/prophet/diagnostics.py#L365

This makes it impossible for a user to create their own performance measure calculator for inclusion without modifying the package code (monkey patching).

I propose a decorator-based, performance-measure registration system to decouple the list of potential performance measures from `diagnostics.performance_metrics()`. This would improve maintainability of the Prophet code in the case new performance measures are included within the package, and would allow users to register their own performance measures without monkey patching the package.

For example:

Existing:
```
def mse(df, w):
se = (df['y'] - df['yhat']) ** 2
if w < 0:
return pd.DataFrame({'horizon': df['horizon'], 'mse': se})
return rolling_mean_by_h(
x=se.values, h=df['horizon'].values, w=w, name='mse'
)
```
Would become
```
@performance_metric
def mse(df, w):
se = (df['y'] - df['yhat']) ** 2
if w < 0:
return pd.DataFrame({'horizon': df['horizon'], 'mse': se})
return rolling_mean_by_h(
x=se.values, h=df['horizon'].values, w=w, name='mse'
)
```

With:
```
PERFORMANCE_METRICS=dict()
def performance_metric(func):
PERFORMANCE_METRICS[func.__name__] = func
return func
```
and with changes to `diagnostics.performance_metrics()` to reference the new PERFORMANCE_METRICS dictionary.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.