Difference in outputs from cross_validation function between daily and weekly data
- 主要言語
- Python
- スター
- 20.4k
- フォーク
- 4.6k
- 平均マージ
- 19時間 52分
- マージ済み PR(30日)
- 1
説明
Hi all!
I noticed that the metrics results from the _cross_validation_ and _performance_metrics_ functions were not varying depending on the combination of hyperparameters. I'm coding in pyspark and using [applyInPandas](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.GroupedData.applyInPandas.html) to forecast a **weekly** time series in pyspark data frame format.
So I got an example from [medium](https://higee.medium.com/parallel-model-training-with-prophet-and-spark-5f40be750f97) with a **daily** time series and the outputs from _cross_validation_ and _performance_metrics_ functions are varying.

For a test, I resampled the same data from tutorial to weekly timestamp, and the outputs did not vary.

The function that I'm using to run cross validation using prophet is:
```python
# based on: https://higee.medium.com/parallel-model-training-with-prophet-and-spark-5f40be750f97
def run_cross_validation(keys, df):
# sorting through timestamp
df = df.sort_values('ds')
# setting keys
item, seasonality_mode, changepoint_prior_scale, seasonality_prior_scale = keys
# model instance
model = Prophet(
seasonality_mode=seasonality_mode,
changepoint_prior_scale=changepoint_prior_scale,
seasonality_prior_scale=seasonality_prior_scale
)
# training model
model.fit(df)
# cross-validation for daily data
df_cv = cross_validation(
model,
horizon=pd.Timedelta('{} W'.format(8)),
parallel='processes'
)
# getting performance metrics
df_perf_metrics = performance_metrics(
df_cv,
metrics=['mse'],
rolling_window=1
)
metric = np.mean(df_perf_metrics['mse'].values[:3])
df_perf_metrics_mse = pd.DataFrame(
data=[metric],
columns=['mse']
)
mapping = {
'item': item,
'seasonality_mode': seasonality_mode,
'changepoint_prior_scale': changepoint_prior_scale,
'seasonality_prior_scale': seasonality_prior_scale
}
df_perf_metrics_mse = df_perf_metrics_mse.assign(**mapping)
cols = [
'item',
'mse',
'seasonality_mode',
'changepoint_prior_scale',
'seasonality_prior_scale'
]
return df_perf_metrics_mse[cols]
df_param = spark.createDataFrame([
(
['multiplicative', 'additive'],
[0.001, 0.005, 0.01, 0.1, 0.3, 0.5],
[0.01, 0.05, 0.1, 1.0, 5.0, 10.0],
)],
[
'seasonality_mode',
'changepoint_prior_scale',
'seasonality_prior_scale'
]
)
for col in df_param.columns:
df_param = df_param.withColumn(col, explode(col))
df_input = spark.createDataFrame(df) # pandas data frame to pyspark data frame
df_input_param = df_input.crossJoin(df_param)
cross_validation_schema = StructType([
StructField('item', StringType()),
StructField('mse', FloatType()),
StructField('seasonality_mode', StringType()),
StructField('changepoint_prior_scale', FloatType()),
StructField('seasonality_prior_scale', FloatType())
])
group_by_col = [
'item',
'seasonality_mode',
'changepoint_prior_scale',
'seasonality_prior_scale'
]
df_cross_validation = df_input_param.groupBy(group_by_col).applyInPandas(func=run_cross_validation, schema=cross_validation_schema)
```
Does anyone know what I'm doing wrong?
コントリビューションガイド
評価
この issue はまだ評価されていません。