deepmodeling / deepmodeling/Uni-Mol

function cal_nan_metric() is not stable in unimol_tools

Open
#154 3 comments 0 reactions 1 assignee Claimed by @HongshuaiWang1 View on GitHub
Dominant language
Python
Stars
1.2k
Forks
181
PR merge metrics
No merged PRs in 30d

Description

```
>>> clf = MolTrain(task='multilabel_regression', data_type='molecule', epochs=10, batch_size=96, metrics='rmse')
>>> pred = clf.fit(data = data)
2023-09-07 15:39:32 | unimol_tools/data/datascaler.py | 84 | INFO | Uni-Mol(QSAR) | Auto select power transformer.
2023-09-07 15:39:32 | unimol_tools/data/datascaler.py | 84 | INFO | Uni-Mol(QSAR) | Auto select power transformer.
2023-09-07 15:39:48 | unimol_tools/train.py | 88 | INFO | Uni-Mol(QSAR) | Output directory already exists: ./exp
2023-09-07 15:39:48 | unimol_tools/train.py | 89 | INFO | Uni-Mol(QSAR) | Warning: Overwrite output directory: ./exp
Traceback (most recent call last):
File "", line 1, in
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/train.py", line 54, in fit
self.trainer = Trainer(save_path=self.save_path, **self.config)
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/tasks/trainer.py", line 36, in __init__
self.metrics = Metrics(self.task, self.metrics_str)
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/utils/metrics.py", line 120, in __init__
self.metric_dict = self._init_metrics(self.task, metrics_str, **params)
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/utils/metrics.py", line 132, in _init_metrics
raise ValueError('Unknown metric: {}'.format(key))
ValueError: Unknown metric: rmse
>>> clf = MolTrain(task='multilabel_regression', data_type='molecule', epochs=10, batch_size=128, )
>>> pred = clf.fit(data = data)
2023-09-07 15:40:52 | unimol_tools/data/datascaler.py | 84 | INFO | Uni-Mol(QSAR) | Auto select power transformer.
2023-09-07 15:40:52 | unimol_tools/data/datascaler.py | 84 | INFO | Uni-Mol(QSAR) | Auto select power transformer.
2023-09-07 15:41:09 | unimol_tools/train.py | 88 | INFO | Uni-Mol(QSAR) | Output directory already exists: ./exp
2023-09-07 15:41:09 | unimol_tools/train.py | 89 | INFO | Uni-Mol(QSAR) | Warning: Overwrite output directory: ./exp
2023-09-07 15:41:10 | unimol_tools/models/unimol.py | 114 | INFO | Uni-Mol(QSAR) | Loading pretrained weights from /share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/weights/mol_pre_all_h_220816.pt
2023-09-07 15:41:10 | unimol_tools/models/nnmodel.py | 103 | INFO | Uni-Mol(QSAR) | start training Uni-Mol:unimolv1
2023-09-07 15:42:46 | unimol_tools/tasks/trainer.py | 169 | INFO | Uni-Mol(QSAR) | Epoch [1/10] train_loss: 1.3208, val_loss: 1.9777, val_mse: 0.0016, lr: 0.000093, 95.6s
2023-09-07 15:44:21 | unimol_tools/tasks/trainer.py | 169 | INFO | Uni-Mol(QSAR) | Epoch [2/10] train_loss: 1.3285, val_loss: 1.8310, val_mse: 0.0015, lr: 0.000082, 94.5s
2023-09-07 15:45:57 | unimol_tools/tasks/trainer.py | 169 | INFO | Uni-Mol(QSAR) | Epoch [3/10] train_loss: 1.3303, val_loss: 1.8310, val_mse: 0.0015, lr: 0.000072, 95.6s
2023-09-07 15:47:31 | unimol_tools/tasks/trainer.py | 169 | INFO | Uni-Mol(QSAR) | Epoch [4/10] train_loss: 1.3294, val_loss: 1.8310, val_mse: 0.0015, lr: 0.000062, 94.0s
val: 99%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 113/114 [00:18<00:00, 6.00it/s, Epoch=Epoch 5/10, loss=nan]Traceback (most recent call last):
File "", line 1, in
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/train.py", line 56, in fit
self.model.run()
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/models/nnmodel.py", line 120, in run
_y_pred = self.trainer.fit_predict(
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/tasks/trainer.py", line 157, in fit_predict
y_preds, val_loss, metric_score = self.predict(
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/tasks/trainer.py", line 254, in predict
metric_score = self.metrics.cal_metric(
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/utils/metrics.py", line 197, in cal_metric
return self.cal_reg_metric(label, predict, nan_value)
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/utils/metrics.py", line 175, in cal_reg_metric
res_dict[metric_type] = nan_metric(label, predict)
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/utils/metrics.py", line 173, in nan_metric
def nan_metric(label, predict): return cal_nan_metric(
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/unimol_tools-1.0.0-py3.10.egg/unimol_tools/utils/metrics.py", line 49, in cal_nan_metric
result.append(metric_func(
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/sklearn/utils/_param_validation.py", line 211, in wrapper
return func(*args, **kwargs)
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/sklearn/metrics/_regression.py", line 474, in mean_squared_error
y_type, y_true, y_pred, multioutput = _check_reg_targets(
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/sklearn/metrics/_regression.py", line 101, in _check_reg_targets
y_pred = check_array(y_pred, ensure_2d=False, dtype=dtype)
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/sklearn/utils/validation.py", line 959, in check_array
_assert_all_finite(
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/sklearn/utils/validation.py", line 124, in _assert_all_finite
_assert_all_finite_element_wise(
File "/share/home/wangjiawen/anaconda3/envs/unimol/lib/python3.10/site-packages/sklearn/utils/validation.py", line 173, in _assert_all_finite_element_wise
raise ValueError(msg_err)
ValueError: Input contains NaN.
```

- metrics function has NaN value:
https://github.com/dptech-corp/Uni-Mol/blob/b640dda4397f276d0873d74b332d7e4d1e9454f6/unimol_tools/unimol_tools/utils/metrics.py#L30-L51

- **multilabel_regression** not support `rmse` metrics as tutorial on bohrium:
https://github.com/dptech-corp/Uni-Mol/blob/b640dda4397f276d0873d74b332d7e4d1e9454f6/unimol_tools/unimol_tools/utils/metrics.py#L100-L113
![image](https://github.com/dptech-corp/Uni-Mol/assets/48350636/c2076def-66d6-4161-8d62-fff662668f2d)

Last, **multilabel_regression**'s loss is hard to decrease.

Looking for solution, thank you! 👀

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.