common-workflow-language / common-workflow-language/cwltool
Piping all streams to common log file
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
I would like to accomplish the equivalent of the following:
`cwltool app.cwl job.yml 2>&1 | tee run.log`
When I run the above CLI code, I get all streams logged in both the console and the log file without even modifying the original app.cwl definition. I am not sure if there is any way to do something similar directly with the runtime execution classes of `cwltool`? I want to use the code directly instead of the CLI (as you know I'm employing cwltool to run https://github.com/crim-ca/weaver).
For the moment, I have only managed to capture `cwltool`'s log via python logger named the same, and capture the executed app's logs using the CWL `stderr` and `stdout` definitions into independent files. The problem is that I would like to have all of those together so that I can report a continuous execution flow of output/errors to the user.
If CWL `stderr` and `stdout` could be pointing to a same file without concurrent access, I could probably manually concatenate the `cwltool`'s and the app's log (stderr/stdout combined), but I have no way currently to safely combine the streams of the app.
Furthermore, CWL `stderr` and `stdout` are very limiting as I cannot see the outputs anymore in the console as they are being captured completely to the log files. If these could accept some kind of pipe notation instead, this would offer more control over the desired reporting.
Slightly related to #364.
Contributor guide
Assessment
This issue has not been assessed yet.