common-workflow-language / common-workflow-language/cwltool
regression of cwltool performance by 5-7 minutes around 1st of june
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
Our PGAPX tool (ncbi/pgap on GitHub) uses cwltool as a CWL runner. We noticed that around 1st of June this year or TeamCity builds are running 5-7 minutes longer.
We tracked the issue to slow down in transition between calling a workflow and command line tool. For example:
```
[2023-09-29 01:38:51] DEBUG [job cluster_blastp_wnode_3] initial work dir {}
[2023-09-29 01:40:22] INFO [job cluster_blastp_wnode_3] /pgap/output/debug/tmp-outdir/kf1wjucm$ cluster_blastp_wnode \
```
See 1.5 min delay that used to be significantly smaller. The cwltool versions of "slow" and "fast" runs are correspondingly:
```
$ docker run -it ncbi/pgap-dev:2023-05-15.build4593 cwltool --version
/root/venv/bin/cwltool 3.1.20230513155734
$ docker run -it ncbi/pgap-dev:2023-06-06.build4641 cwltool --version
/root/venv/bin/cwltool 3.1.20230601100705
```
We directly tested that reversion of cwltool version to May value fixes the performance degradation issue.
Contributor guide
Assessment
This issue has not been assessed yet.