microsoft / microsoft/qlib

PortAnaRecord not supporting highfreq data

Open
#341 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
48.7k
Forks
7.7k
PR merge metrics
No merged PRs in 30d

Description

[32154:MainThread](2021-03-15 01:13:58,756) WARNING - qlib.Initialization - [config.py:292] - redis connection failed(host=127.0.0.1 port=6379), cache will not be used!
[32154:MainThread](2021-03-15 01:13:58,760) INFO - qlib.Initialization - [__init__.py:46] - qlib successfully initialized based on client settings.
[32154:MainThread](2021-03-15 01:13:58,760) INFO - qlib.Initialization - [__init__.py:47] - data_path=xxxxxxxxxxxxxxxxxxxxx/qlib_data/hour
[32154:MainThread](2021-03-15 01:16:29,323) INFO - qlib.timer - [log.py:77] - Time cost: 148.341s | Loading data Done
[32154:MainThread](2021-03-15 01:16:44,864) INFO - qlib.timer - [log.py:77] - Time cost: 11.575s | DropnaLabel Done
[32154:MainThread](2021-03-15 01:16:48,886) INFO - qlib.timer - [log.py:77] - Time cost: 4.021s | CSRankNorm Done
[32154:MainThread](2021-03-15 01:16:48,886) INFO - qlib.timer - [log.py:77] - Time cost: 19.562s | fit & process data Done
[32154:MainThread](2021-03-15 01:16:48,886) INFO - qlib.timer - [log.py:77] - Time cost: 167.905s | Init data Done
[32154:MainThread](2021-03-15 01:17:30,866) INFO - qlib.workflow - [expm.py:248] - No tracking URI is provided. Use the default tracking URI.
[32154:MainThread](2021-03-15 01:17:30,869) INFO - qlib.workflow - [expm.py:284] - <mlflow.tracking.client.MlflowClient object at 0x2b1a7bf54940>
[32154:MainThread](2021-03-15 01:17:30,997) INFO - qlib.workflow - [exp.py:182] - Experiment 1 starts running ...
[32154:MainThread](2021-03-15 01:17:38,976) INFO - qlib.workflow - [recorder.py:270] - Recorder fcbe541714924bf59c8d72cdc582fb4f starts running under Experiment 1 ...
[32154:MainThread](2021-03-15 02:06:02,266) INFO - qlib.workflow - [record_temp.py:126] - Signal record 'pred.pkl' has been saved as the artifact of the Experiment 1
[32154:MainThread](2021-03-15 02:06:02,607) INFO - qlib.backtest caller - [__init__.py:148] - Create new exchange
[32154:MainThread](2021-03-15 02:06:02,646) ERROR - qlib.workflow - [utils.py:35] - An exception has been raised[ValueError: calendar not exists for freq day].
  File "./workflow-highfreq.py", line 45, in <module>
    par.generate()
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/workflow/record_temp.py", line 243, in generate
    report_dict = normal_backtest(pred_score, strategy=self.strategy, **self.backtest_config)
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/contrib/backtest/__init__.py", line 303, in backtest
    trade_exchange = get_exchange(pred, **ex_args)
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/contrib/backtest/__init__.py", line 158, in get_exchange
    dates = np.append(dates, get_date_range(dates[-1], left_shift=1, right_shift=shift))
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/utils/__init__.py", line 498, in get_date_range
    start = get_date_by_shift(trading_date, left_shift, future=future)
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/utils/__init__.py", line 518, in get_date_by_shift
    cal = D.calendar(future=future)
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/data/data.py", line 928, in calendar
    return Cal.calendar(start_time, end_time, freq, future=future)
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/data/data.py", line 534, in calendar
    _calendar, _calendar_index = self._get_calendar(freq, future)
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/data/data.py", line 120, in _get_calendar
    _calendar = np.array(self.load_calendar(freq, future))
  File "xxxxxxxxxxxxxxxxxxxxx/.venv/lib/python3.7/site-packages/pyqlib-0.6.3.99-py3.7-linux-x86_64.egg/qlib/data/data.py", line 529, in load_calendar
    raise ValueError("calendar not exists for freq " + freq)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with workflow-highfreq.py and reproduce the failure reported by par.generate(). Trace the backtest path through qlib/workflow/record_temp.py, qlib/contrib/backtest/init.py, qlib/utils/init.py, and qlib/data/data.py. Done means high-frequency data can complete this backtest without the missing day-calendar error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, fintech-quant
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.