microsoft / microsoft/qlib

The benchmark 000300 does not exist. Please provide the right benchmark

Open
#2,205 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
48.7k
Forks
7.7k
PR merge metrics
No merged PRs in 30d

Description

🐛 Bug Description

This error occurs when training custom data using the code in the examples that modifies workflow_by_customer_code.py; even if you set benchmark to None, the problem is still there.

To Reproduce

Steps to reproduce the behavior:

  1. prepare your data
  2. run workflow_by_customer_code.py, be provided later
  3. python3 examples/workflow_by_customer_code.py

Expected Behavior

This error will be gone.

Screenshot

Image

Environment

Note: User could run cd scripts && python collect_info.py all under project directory to get system information
and paste them here directly.

  • Qlib version: 0.9.8.dev31
  • Python version: 3.12.13
  • OS (MacOS):

Additional Notes

workflow_by_customer_code.py script

import qlib
import pandas as pd
from qlib.constant import REG_CN
from qlib.utils import init_instance_by_config, flatten_dict
from qlib.workflow import R
from qlib.workflow.record_temp import SignalRecord, PortAnaRecord, SigAnaRecord
from qlib.tests.config import CSI300_BENCH

TASK = {
    "model": {
        "class": "LGBModel",
        "module_path": "qlib.contrib.model.gbdt",
        "kwargs": {
            "loss": "mse",
            "colsample_bytree": 0.8879,
            "learning_rate": 0.0421,
            "subsample": 0.8789,
            "lambda_l1": 205.6999,
            "lambda_l2": 580.9768,
            "max_depth": 8,
            "num_leaves": 210,
            "num_threads": 10,
        },
    },
    "dataset": {
        "class": "DatasetH",
        "module_path": "qlib.data.dataset",
        "kwargs": {
            "handler": {
                "class": "Alpha158",
                "module_path": "qlib.contrib.data.handler",
                "kwargs": {
                    "start_time": "2020-01-01",
                    "end_time": "2026-05-07",
                    "fit_start_time": "2020-01-01",
                    "fit_end_time": "2024-12-31",
                    "instruments": "all",
                },
            },
            "segments": {
                "train": ("2020-01-01", "2024-12-31"),
                "valid": ("2025-01-01", "2025-06-30"),
                "test": ("2025-07-01", "2026-05-07"),
            },
        },
    },
}

if __name__ == "__main__":
    # provider_uri = "~/.qlib/qlib_data/cn_data"
    provider_uri = "~/qlib/examples/data/qlib_data"
    qlib.init(provider_uri=provider_uri, region=REG_CN)

    model = init_instance_by_config(TASK["model"])
    dataset = init_instance_by_config(TASK["dataset"])

    port_analysis_config = {
        "executor": {
            "class": "SimulatorExecutor",
            "module_path": "qlib.backtest.executor",
            "kwargs": {
                "time_per_step": "day",
                "generate_portfolio_metrics": True,
            },
        },
        "strategy": {
            "class": "TopkDropoutStrategy",
            "module_path": "qlib.contrib.strategy.signal_strategy",
            "kwargs": {
                "signal": (model, dataset),
                "topk": 5,
                "n_drop": 2,
            },
        },
        "backtest": {
            "start_time": "2025-07-01",
            "end_time": "2026-05-07",
            "account": 10000000,
            "benchmark": CSI300_BENCH,
            "exchange_kwargs": {
                "freq": "day",
                "limit_threshold": 0.095,
                "deal_price": "close",
                "open_cost": 0.0005,
                "close_cost": 0.0015,
                "min_cost": 5,
            },
        },
    }

    with R.start(experiment_name="Strategy_Research"):
        R.log_params(**flatten_dict(TASK))
        
        model.fit(dataset)
        R.save_objects(**{"params.pkl": model})

        recorder = R.get_recorder()
        sr = SignalRecord(model, dataset, recorder)
        sr.generate()

        sar = SigAnaRecord(recorder)
        sar.generate()

        par = PortAnaRecord(recorder, port_analysis_config, "day")
        par.generate()

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/workflow_by_customer_code.py and trace how the CSI300_BENCH value reaches the backtest configuration. Reproduce the failure with the provided script and custom-data setup, then verify the workflow no longer requests the missing benchmark and that the backtest completes successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.