common-workflow-language / common-workflow-language/cwltool

No longer able to parse json files within outputEval

Open
#977 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

## Workflow Code
```
cwlVersion: v1.0
class: CommandLineTool
requirements:
DockerRequirement: {dockerPull: "python"}
InlineJavascriptRequirement: {}
baseCommand:
- python
- -c
- "import json; json.dump({'key1': 'foobar', 'key2': 2}, open('out.json', 'w'))"
inputs: []
outputs:
output:
type:
type: record
fields:
- name: key1
type: string
- name: key2
type: int
outputBinding:
loadContents: true
glob: "out.json"
outputEval: |
${
var res = JSON.parse(self[0].contents)
console.log(res);
return res;
}
```

## Expected Behavior
Previously, I have parsed the json output of a CommandLineTool with `JSON.parse(self[0].contents)`. The above workflow runs properly in, e.g., `v1.0.20180525185854`. It returns:

```
Javascript console output:
----------------------------------------
[log] { key1: 'foobar', key2: 2 }
----------------------------------------
[job wf.cwl] completed success
{
"output": {
"key2": 2,
"key1": "foobar"
}
}
Final process status is success
```

## Actual Behavior
Now, I get a validation error:

```
Error validating output record. the `output` field is not valid because
* the `key1` field is not valid because
the value is not string
* the `key2` field is not valid because
`[]` is not int
in {
"output": {
"key1": [],
"key2": []
}
}
```

## Full Traceback
```
[job wf.cwl] /private/tmp/docker_tmp5xw1jlmb$ docker \
run \
-i \
--volume=/private/tmp/docker_tmp5xw1jlmb:/uYEajn:rw \
--volume=/private/var/folders/x0/0s7bygdj26db4gd20fx6t7k00000gn/T/tmpk1xyygwi:/tmp:rw \
--workdir=/uYEajn \
--read-only=true \
--user=501:20 \
--rm \
--env=TMPDIR=/tmp \
--env=HOME=/uYEajn \
--cidfile=/minimal_failing_example/20181115133418-855907.cid \
python \
python \
-c \
'import json; json.dump({'"'"'key1'"'"': '"'"'foobar'"'"', '"'"'key2'"'"': 2}, open('"'"'out.json'"'"', '"'"'w'"'"'))'
Running with support for javascript console in expressions (DO NOT USE IN PRODUCTION)
Javascript console output:
----------------------------------------
/python3.7/site-packages/cwltool/sandboxjs.py:342: FutureWarning: Possible nested set at position 2
r'^[[](?:log|err)[]].*$', stderr, flags=re.MULTILINE)))
[log] { key1: 'foobar', key2: 2 }
----------------------------------------
[job wf.cwl] Job error:
Error validating output record. the `output` field is not valid because
* the `key1` field is not valid because
the value is not string
* the `key2` field is not valid because
`[]` is not int
in {
"output": {
"key1": [],
"key2": []
}
}
[job wf.cwl] completed permanentFail
{}
Final process status is permanentFail
```

## Environment
* cwltool version: 1.0.20181102182747

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.