unit8co / unit8co/darts

[QUESTION] Trouble creating the right dataset for historical backtest with updating covariates

Open
#2,575 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
9.5k
Forks
1k
Avg merge
5d 22h
Merged PRs (30d)
14

Description

I have data that is of 1 hour granularity. I would like to create a model that forecasts the next 7 days of values every 24 hours. My future covariates are updated every day.

Production Example:
Current time: 2024-01-01 23:00:00
Prediction horizon (7 days): 2024-01-02 00:00:00 to 2024-01-08 23:00:00
Target values: Latest target I have is until 2024-01-01 23:00:00
Future covariates: until 2024-01-08 23:00:00

I would like to predict each day's values using only the target_lag from the previous day and the future_cov from the same day. Hence, I define my model as such:

lgbm_model = LightGBMModel(
    lags=list(range(-24, 0)),
    lags_future_covariates=list(range(0, 24)),
    output_chunk_length=24,
    n_jobs=-1,
    random_state=42,
    multi_models=True,
    verbose=0,
)

When trying to create the historical backtest, this is the implementation I came up with:

for date in pd.date_range(split_date, end_date - timedelta(days = 7)):
    print(f'As of {date} predicting from: {date + relativedelta(hours = 1)} to {date+relativedelta(days = 7)}')
    target_series_train = target_series[start_date: date]
    future_cov_train = future_cov_series[start_date: date]
    
    lgbm_model.fit(
        series = target_series_train,
        future_covariates = future_cov_train
    )
    
    data_df_test_sample = best_guess_df[best_guess_df.forecast_date == date.date()]
    target_series_test = target_series_train[-24:]
    future_cov_series_test = TimeSeries.from_dataframe(data_df_test_sample[future_cov])
    forecast_results = lgbm_model.predict(n=168, series=target_series_test, future_covariates=future_cov_series_test).pd_dataframe()
    break

I then tried printing out the training set using

lagged_training_data = create_lagged_training_data(
            target_series=target_series_train,
            past_covariates=None,
            future_covariates=future_cov_series_train,
            output_chunk_shift=0,
            lags=target_lags,
            lags_past_covariates=None,
            lags_future_covariates=future_cov_lags,
            output_chunk_length=24,
            multi_models=True,
            uses_static_covariates=False
        )

From the training set, it appears that when predicting for the first day, because the target_lag is defined as [-24, -1], the target_lag values used from the second hour onwards does not exist. For example, when trying to predict for the target at 5am, the target_lag-1 is the value at 4am (which does not exist in production). The latest target value I have is from 23:00 from the previous day. How would I define my model such that it always uses the 24 target values from the day before to predict all hours on the day of prediction?

Example dataset:
image

What I want the training set to be (Assuming output chunk length = 2):
image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the LightGBMModel lag handling and the create_lagged_training_data entry point; reproduce the reported hourly training rows using the example dataset and lag settings. Determine whether the requested previous-day target window is supported by the current API, and document or test the expected training behavior when the issue is resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.