Optional omitted Complex Input with default format is generated errorneously (?)
- Dominant language
- Python
- Stars
- 186
- Forks
- 117
- Avg merge
- 11d 20h
- Merged PRs (30d)
- 1
Description
# Description
@cehbrecht @tomkralidis @jachym
I would like to better understand the procedure of handling inputs (how they get generated) for the following specific use case.
Given a process that has the following inputs definition :
```xml
[...]
dataset
Dataset
Enter a URL pointing to a NetCDF file (optional)
application/x-netcdf
application/x-netcdf
dataset_opendap
Remote OpenDAP Data URL
Or provide a remote OpenDAP data URL, for example: http://my.opendap/thredds/dodsC/path/to/file.nc
string
[...]
```
When I submit an execution with _**only**_ input `dataset_opendap` provided with some URL string, the `_handler(self, request, response)` method of the process that ends up being called contains the following `request.inputs`:
```python
request.inputs = {
'dataset': [],
'dataset_opendap': deque([], maxlen=100)
}
```
My execution XML does not contain `dataset`, so it gets generated somehow by default following parsing.
```xml
ncdump
dataset_opendap
http://localhost8001/ows/proxy/thredds/dodsC/birdhouse/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc
output
```
I'm trying to understand why the `dataset` input even gets generated in `request.inputs` following parsing since it is omitted completely from the request.
This input is causing me problems, because I need to do some post-processing to convert PyWPS inputs into my package definitions.
Is there some way that I need to employ to detect omitted inputs to discard them explicitly vs real inputs with submitted data?
Is there some flag that I would guarantee me that this input is only the default definition and does not contain any actual data?
I cannot rely on `data` field to detect omitted inputs because it gets filled by the "default format" `application/x-netcdf`, which could be submitted real data contents:
```json
{"mimeType": "application/x-netcdf", "encoding": null, "schema": null, "maximumMegabytes": null, "default": true}
```
The only (very hackish/unreliable) field I could use to detect inputs to drop is `file` which contains a reference to `{workdir}/input` instead of `{workdir}/input_{uuid}`.
Any better guidance would be greatly appreciated.
Expand this to see full details contents of requests.inputs["dataset"]
{ComplexInput}
_data_format = {Format}
_encoding = {NoneType} None
_extension = {NoneType} None
_mime_type = {str} 'application/x-netcdf'
_schema = {NoneType} None
encoding = {str} ''
extension = {str} ''
json = {dict} {'mime_type': 'application/x-netcdf', 'encoding': '', 'schema': '', 'extension': ''}
mime_type = {str} 'application/x-netcdf'
schema = {str} ''
_default = {dict} {'mimeType': 'application/x-netcdf', 'encoding': None, 'schema': None, 'maximumMegabytes': None, 'default': True}
_default_type = {int} 3
_iohandler = {DataHandler}
_data = {dict}
_file = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee/input'
_ref = {weakref}
_stream = {NoneType} None
base64 = {str} 'Traceback (most recent call last):\n File "/opt/pycharm-pro/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 178, in _getPyDictionary\n attr = getattr(var, n)\n File "/home/francis/dev/miniconda/envs/weaver-py3/lib/python3.7/site-pa
data = {dict} {'mimeType': 'application/x-netcdf', 'encoding': None, 'schema': None, 'maximumMegabytes': None, 'default': True}
file = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee/input'
mem = {NoneType} None
post_data = {str} 'Traceback (most recent call last):\n File "/opt/pycharm-pro/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 178, in _getPyDictionary\n attr = getattr(var, n)\n File "/home/francis/dev/miniconda/envs/weaver-py3/lib/python3.7/site-pa
prop = {str} 'data'
size = {int} 0
stream = {StringIO} <_io.StringIO object at 0x7f140d8aa690>
url = {str} 'file:///tmp/weaver-hybrid/pywps_process_pw40isee/input'
_supported_formats = {tuple}
_workdir = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee'
abstract = {str} 'Enter a URL pointing to a NetCDF file (optional)'
as_reference = {bool} False
base64 = {str} 'Traceback (most recent call last):\n File "/opt/pycharm-pro/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 178, in _getPyDictionary\n attr = getattr(var, n)\n File "/home/francis/dev/miniconda/envs/weaver-py3/lib/python3.7/site-pa
data = {dict} {'mimeType': 'application/x-netcdf', 'encoding': None, 'schema': None, 'maximumMegabytes': None, 'default': True}
data_format = {Format}
data_set = {bool} True
extension = {str} ''
file = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee/input'
identifier = {str} 'dataset'
inpt = {dict} {}
json = {dict} {'identifier': 'dataset', 'title': 'Dataset', 'abstract': 'Enter a URL pointing to a NetCDF file (optional)', 'keywords': [], 'metadata': [], 'type': 'complex', 'data_format': {'mime_type': 'application/x-netcdf', 'encoding': '', 'schema': '', 'extension': ''}, 'asreference': False, 'supported_formats': [{'mime_type': 'application/x-netcdf', 'encoding': '', 'schema': '', 'extension': ''}], 'workdir': '/tmp/weaver-hybrid/pywps_process_pw40isee', 'mode': 0, 'min_occurs': 0, 'max_occurs': 100, 'translations': None, 'data': "", 'mimetype': 'application/x-netcdf'}
keywords = {list} []
max_occurs = {int} 100
metadata = {list} []
method = {str} ''
min_occurs = {int} 0
post_data = {str} 'Traceback (most recent call last):\n File "/opt/pycharm-pro/plugins/python/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 178, in _getPyDictionary\n attr = getattr(var, n)\n File "/home/francis/dev/miniconda/envs/weaver-py3/lib/python3.7/site-pa
prop = {str} 'data'
size = {int} 0
source_type = {int} 3
stream = {StringIO}
supported_formats = {tuple}
title = {str} 'Dataset'
translations = {NoneType} None
url = {str} 'file:///tmp/weaver-hybrid/pywps_process_pw40isee/input'
uuid = {NoneType} None
valid_mode = {int} 0
workdir = {str} '/tmp/weaver-hybrid/pywps_process_pw40isee'
# Environment
- operating system:
- Python version: 3.7
- PyWPS version: tried with both with 4.4.5 and 4.5.0, same result
- source/distribution
- [x] git clone
- [ ] Debian
- [x] PyPI
- [ ] zip/tar.gz
- [ ] other (please specify):
- web server
- [ ] Apache/mod_wsgi
- [ ] CGI
- [x] other (please specify): `gunicorn` with `Pyramid` WebApp that nests PyWPS's service
# Steps to Reproduce
Using this process:
https://github.com/bird-house/hummingbird/blob/master/hummingbird/processes/wps_ncdump.py
It is executed indirectly by Weaver using this definition:
https://github.com/crim-ca/weaver/blob/4.1.0/weaver/processes/wps_package.py#L758
Contributor guide
Assessment
This issue has not been assessed yet.