microsoft / microsoft/qlib

Qlib RL 是否不支持一分鐘數據訓練嗎?

Open
#1,779 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
48.7k
Forks
7.7k
PR merge metrics
No merged PRs in 30d

Description

(qqlib) C:\Users\man-pc\qlib>python examples/rl_order_execution/scripts/gen_pickle_data.py -c examples/rl_order_execution/scripts/pickle_data_config.yml
[18380:MainThread](2024-04-23 08:22:19,538) INFO - qlib.Initialization - [config.py:416] - default_conf: client.
[18380:MainThread](2024-04-23 08:22:20,609) INFO - qlib.Initialization - [init.py:74] - qlib successfully initialized based on client settings.
[18380:MainThread](2024-04-23 08:22:20,609) INFO - qlib.Initialization - [init.py:76] - data_path={'1min': WindowsPath('C:/Users/man-pc/qlib/data/bin')}
[18380:MainThread](2024-04-23 08:22:20,610) INFO - qlib.HighFreqProvider - [highfreq_provider.py:144] - [qlib.contrib.data.highfreq_provider]Generating dataset
[18380:MainThread](2024-04-23 08:22:24,047) INFO - qlib.timer - [log.py:127] - Time cost: 2.972s | Loading data Done
C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\contrib\data\highfreq_processor.py:52: RuntimeWarning: Mean of empty slice
self.feature_mean = np.nanmean(df_values)
C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\numpy\lib\nanfunctions.py:1878: RuntimeWarning: Degrees of freedom <= 0 for slice.
var = nanvar(a, axis=axis, dtype=dtype, out=out, ddof=ddof,
[18380:MainThread](2024-04-23 08:22:24,051) ERROR - qlib.workflow - [utils.py:41] - An exception has been raised[ValueError: zero-size array to reduction operation fmax which has no identity].
File "examples/rl_order_execution/scripts/gen_pickle_data.py", line 30, in
feature = provider._gen_dataframe(deepcopy(provider.feature_conf))
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\contrib\data\highfreq_provider.py", line 147, in gen_dataframe
dataset = init_instance_by_config(config)
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\utils\mod.py", line 174, in init_instance_by_config
return klass(**cls_kwargs, **try_kwargs, **kwargs)
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\data\dataset_init
.py", line 119, in init
self.handler: DataHandler = init_instance_by_config(handler, accept_types=DataHandler)
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\utils\mod.py", line 174, in init_instance_by_config
return klass(**cls_kwargs, **try_kwargs, **kwargs)
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\contrib\data\highfreq_handler.py", line 134, in init
super().init(
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\data\dataset\handler.py", line 468, in init
super().init(instruments, start_time, end_time, data_loader, **kwargs)
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\data\dataset\handler.py", line 100, in init
self.setup_data()
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\data\dataset\handler.py", line 619, in setup_data
self.fit_process_data()
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\data\dataset\handler.py", line 487, in fit_process_data
self.process_data(with_fit=True)
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\data\dataset\handler.py", line 551, in process_data
_infer_df = self._run_proc_l(_infer_df, self.infer_processors, with_fit=with_fit, check_for_infer=True)
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\data\dataset\handler.py", line 498, in _run_proc_l
proc.fit(df)
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\qlib\contrib\data\highfreq_processor.py", line 58, in fit
np.save(self.feature_save_dir + name + "_vmax.npy", np.nanmax(df_values))
File "<array_function internals>", line 180, in nanmax
File "C:\Users\man-pc\AppData\Local\anaconda3\envs\qqlib\lib\site-packages\numpy\lib\nanfunctions.py", line 476, in nanmax
res = np.fmax.reduce(a, axis=axis, out=out, **kwargs)
ValueError: zero-size array to reduction operation fmax which has no identity

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/rl_order_execution/scripts/gen_pickle_data.py and pickle_data_config.yml, then trace the one-minute data path through qlib/contrib/data/highfreq_provider.py and highfreq_processor.py. Reproduce the command and determine why processing receives an empty array. Done means the one-minute dataset generation either completes successfully or reports the limitation clearly instead of raising the shown reduction error.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data-engineering, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.