Why did I get "killed" when running 1min of alpha360?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 48.7k
- Forks
- 7.7k
- PR merge metrics
- No merged PRs in 30d
Description
###################################
# train model
###################################
data_handler_config = {
"start_time":"2020-09-14",
"end_time":"2021-06-20",
"fit_start_time":"2020-09-20",
"fit_end_time":"2021-05-20",
"freq":'1min',
"instruments": market,
}
task = {
"model": {
"class": "LGBModel",
"module_path": "qlib.contrib.model.gbdt",
"kwargs": {
"loss": "mse",
"colsample_bytree": 0.8879,
"learning_rate": 0.0421,
"subsample": 0.8789,
"lambda_l1": 205.6999,
"lambda_l2": 580.9768,
"max_depth": 8,
"num_leaves": 210,
"num_threads": 20,
},
},
"dataset": {
"class": "DatasetH",
"module_path": "qlib.data.dataset",
"kwargs": {
"handler": {
"class": "Alpha360",
"module_path": "qlib.contrib.data.handler",
"kwargs": data_handler_config,
},
"segments": {
"train": ("2020-09-14", "2020-10-15"),
"valid": ("2020-10-16", "2020-12-20"),
"test": ("2020-12-14", "2021-05-20"),
},
},
},
}
h = Alpha360(**data_handler_config)
alpha360_df_feature = h.fetch(col_set="feature")
alpha360_df_feature = alpha360_df_feature.loc[:,~(alpha360_df_feature == 0.0).all()]
alpha360_df_feature = alpha360_df_feature.loc[~(alpha360_df_feature.eq(0.0).all(1))]
print(alpha360_df_feature)
# model initiaiton
model = init_instance_by_config(task["model"])
dataset = init_instance_by_config(task["dataset"])
# start exp to train model
with R.start(experiment_name="train_model"):
R.log_params(**flatten_dict(task))
model.fit(dataset)
R.save_objects(trained_model=model)
rid = R.get_recorder().id
when I try tu run jupyter notebook, it always showed that python kernel has been restarted.
What's datetime range does it support? I tried one day range, it works fine from 2020-09-14 to 2020-09-15,
Most of feature in alpha360 are zero. this range may a little bit short.
My machine:
Debian 11
RAM 64G
4 processors
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the one-day and longer-range cases with Alpha360.fetch, then compare behavior around model.fit using the supplied LGBModel and DatasetH configuration. Inspect the notebook kernel or process memory while expanding the date range. Done means documenting the supported datetime range and identifying whether the restart is caused by resource use or a data issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- data, machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100