Async processes accumulate until failure with sqlite in-memory
- Dominant language
- Python
- Stars
- 186
- Forks
- 117
- Avg merge
- 11d 20h
- Merged PRs (30d)
- 1
Description
# Description
This relates to running processes in async mode, with `logging.database=sqlite:///:memory:`
When an async process completes, i.e. (pywps_requests.percent_done: 100.0, pywps_requests.status: 4) the `running_count` reported by [dblog](https://github.com/geopython/pywps/blob/034b185c5e3b521d4a6a30fef118c6ab368567a6/pywps/dblog.py#L88) never seems to reflects this.
So if I set `parallelprocesses=5` I can execute 5 successful jobs, however each job increments this running count, which is never decremented on completion, meaning I can only run 5 before all I get is ‘PyWPS Process GetSubset accepted’ response for a process which never runs.
This issue only seems to happen when using in-memory sqlite (i.e. does not occur when supplying my own sqlite db string).
# Environment
- operating system: Debian
- Python version: 3.6.5
- PyWPS version: 4.2.2
- source/distribution
- [x] git clone
- [ ] Debian
- [ ] PyPI
- [ ] zip/tar.gz
- [ ] other (please specify):
- web server
- [ ] Apache/mod_wsgi
- [ ] CGI
- [x] other (please specify): nginx/gunicorn
# Steps to Reproduce
- Run pywps configured with
- `logging.database=sqlite:///:memory:`
- `server.parallelprocesses=5`
- Submit 5 async requests
- i.e. with `storeExecuteResponse=true&status=true`
- should work as expected, then on 6th request:
- [Gets put in the stored queue](https://github.com/geopython/pywps/blob/034b185c5e3b521d4a6a30fef118c6ab368567a6/pywps/app/Process.py#L167)
- despite status being set to [PyWPS Process stored in job queue](https://github.com/geopython/pywps/blob/034b185c5e3b521d4a6a30fef118c6ab368567a6/pywps/app/Process.py#L173), the response document still says ‘PyWPS Process GetSubset accepted’
- nothing else can run now
# Additional Information
This is being run in a docker container from macos host, don’t _think_ it should affect this 🤔
Contributor guide
Assessment
This issue has not been assessed yet.