common-workflow-language / common-workflow-language/cwl-utils

Documentation for importing packed files

Open
#71 10 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
43
Forks
28
Avg merge
4h 7m
Merged PRs (30d)
4

Description

Hello,

Been playing around with how to import a packed cwl json file as a CWL parser object.

Here are my steps so far

## Setup
```python
# Imports
from pathlib import Path
import json
import sys

# Set path
cwl_file_path = Path("/path/to/cwl.packed.json")

# Load file as dict
# Read in the cwl file from a json
with open(cwl_file_path, "r") as cwl_h:
cwl_file_dict = json.load(cwl_h)

# Conditional import based on cwl version
if 'cwlVersion' not in list(cwl_file_dict.keys()):
print("Error - could not get the cwlVersion")
sys.exit(1)
# Import parser based on CWL Version
if cwl_file_dict['cwlVersion'] == 'v1.0':
from cwl_utils import parser_v1_0 as parser
elif cwl_file_dict['cwlVersion'] == 'v1.1':
from cwl_utils import parser_v1_1 as parser
elif cwl_file_dict['cwlVersion'] == 'v1.2':
from cwl_utils import parser_v1_2 as parser
else:
print("Version error. Did not recognise {} as a CWL version".format(yaml_obj["CWLVersion"]))
sys.exit(1)
```

## First attempt:
> Use the load document feature
```python
parser.load_document(cwl_file_dict, cwl_file_path.absolute().as_uri())

SchemaSaladException: Cannot load $import without fileuri
```

## Second attempt
> Convert to string then load
```python
parser.load_document_by_string(json.dumps(cwl_file_dict), cwl_file_path.absolute().as_uri())

ValidationException: - tried _RecordLoader but
Expected a dict
- tried _RecordLoader but
Expected a dict
...
```

## Third attempt
> Convert to yaml then load
```python
# We need to import the ruamel yaml class
from ruamel import yaml
# Dump our dict to a yaml string
cwl_yaml_dump = yaml.round_trip_dump(cwl_file_dict, Dumper=yaml.RoundTripDumper)
# Load yaml
cwl_yaml_load = yaml.round_trip_load(cwl_yaml_dump, preserve_quotes=True)
# Import
parser.load_document_by_yaml(cwl_yaml_load, cwl_file_path.absolute().as_uri())

ValidationException: - tried _RecordLoader but
Expected a dict
- tried _RecordLoader but
Expected a dict
...
```

## Fourth attempt
> Convert just graph to yaml then load
```python
# We need to import the ruamel yaml class
from ruamel import yaml
# Dump our dict to a yaml string
cwl_yaml_dump = yaml.round_trip_dump(cwl_file_dict['$graph'], Dumper=yaml.RoundTripDumper)
# Load yaml
cwl_yaml_load = yaml.round_trip_load(cwl_yaml_dump, preserve_quotes=True)
# Import
parser.load_document_by_yaml(cwl_yaml_load, cwl_file_path.absolute().as_uri())

ValidationException: - tried _RecordLoader but
Expected a dict
- tried _RecordLoader but
Expected a dict
...
```

Is this due to my workflow being a little bit too complicated for the parser and using record schemas?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.