common-workflow-language / common-workflow-language/cwltool

Piping all streams to common log file

Open
#1,287 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

I would like to accomplish the equivalent of the following:
`cwltool app.cwl job.yml 2>&1 | tee run.log`

When I run the above CLI code, I get all streams logged in both the console and the log file without even modifying the original app.cwl definition. I am not sure if there is any way to do something similar directly with the runtime execution classes of `cwltool`? I want to use the code directly instead of the CLI (as you know I'm employing cwltool to run https://github.com/crim-ca/weaver).

For the moment, I have only managed to capture `cwltool`'s log via python logger named the same, and capture the executed app's logs using the CWL `stderr` and `stdout` definitions into independent files. The problem is that I would like to have all of those together so that I can report a continuous execution flow of output/errors to the user.

If CWL `stderr` and `stdout` could be pointing to a same file without concurrent access, I could probably manually concatenate the `cwltool`'s and the app's log (stderr/stdout combined), but I have no way currently to safely combine the streams of the app.

Furthermore, CWL `stderr` and `stdout` are very limiting as I cannot see the outputs anymore in the console as they are being captured completely to the log files. If these could accept some kind of pipe notation instead, this would offer more control over the desired reporting.

Slightly related to #364.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.