nextcloud / nextcloud/workflow_script

`shell_exec(): Unable to execute`

Open
#197 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
89
Forks
19
Avg merge
6h 38m
Merged PRs (30d)
3

Description

Steps to reproduce

  1. Have a workflow that triggers on tag changes and runs a python script (for me it watermarks pdfs and then triggers NC to rescan the directory)
  2. Tag multiple files at once
  3. Wait for the changes to take effect
Expected behaviour

All files are watermarked

Actual behaviour

Only very few files a watermarked, sometimes only 3-4 out of 40. Prepending a random shell sleep to the command does improve this a bit, but not completey. I am wondering if NC itself or this app imposes some limits on background jobs that can be run at the same time or if this might a a limitation of my host. Maybe there is also a way to configure the number of jobs to run in a single cron-job execution?

Workflow Script app

Workflow Script app version: 1.10.0

Server configuration

Operating system: Linux 5.10.0-23-amd64 x86_64

Web server: Apache

Database: MySQL 10.10.5

PHP version: 7.4.33

Nextcloud Version: 25.0.9

Where did you install Nextcloud from: One-click-install via webhost

Signing status:

No errors have been found.

List of activated apps:

Enabled:
  - activity: 2.17.0
  - bruteforcesettings: 2.5.0
  - circles: 25.0.0
  - cloud_federation_api: 1.8.0
  - contacts: 5.3.2
  - dav: 1.24.0
  - documentserver_community: 0.1.13
  - federatedfilesharing: 1.15.0
  - files: 1.20.1
  - files_accesscontrol: 1.15.1
  - files_automatedtagging: 1.15.3
  - files_pdfviewer: 2.6.0
  - files_retention: 1.14.1
  - files_rightclick: 1.4.0
  - files_sharing: 1.17.0
  - files_trashbin: 1.15.0
  - files_versions: 1.18.0
  - firstrunwizard: 2.14.0
  - forms: 3.3.1
  - group_default_quota: 0.1.7
  - groupfolders: 13.1.4
  - logreader: 2.10.0
  - lookup_server_connector: 1.13.0
  - nextcloud_announcements: 1.14.0
  - notifications: 2.13.1
  - oauth2: 1.13.1
  - onlyoffice: 7.8.0
  - password_policy: 1.15.0
  - photos: 2.0.1
  - polls: 4.1.8
  - provisioning_api: 1.15.0
  - quota_warning: 1.17.0
  - ransomware_protection: 1.14.0
  - recommendations: 1.4.0
  - related_resources: 1.0.4
  - serverinfo: 1.15.0
  - settings: 1.7.0
  - support: 1.8.0
  - systemtags: 1.15.0
  - text: 3.6.0
  - theming: 2.0.1
  - twofactor_backupcodes: 1.14.0
  - updatenotification: 1.15.0
  - user_retention: 1.10.0
  - viewer: 1.9.0
  - workflow_script: 1.10.0
  - workflowengine: 2.7.0
Disabled:
  - admin_audit
  - calendar: 4.4.4
  - comments: 1.15.0
  - contactsinteraction: 1.6.0
  - dashboard: 7.5.0
  - encryption
  - federation: 1.15.0
  - files_external
  - privacy: 1.9.0
  - sharebymail: 1.15.0
  - survey_client: 1.13.0
  - suspicious_login
  - twofactor_totp
  - user_ldap
  - user_status: 1.5.0
  - weather_status: 1.5.0

Nextcloud configuration:

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "25.0.9.2",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "theme": "",
        "loglevel": 0,
        "log_type": "file",
        "default_phone_region": "DE",
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_sendmailmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "app_install_overwrite": [
            "apporder",
            "issuetemplate"
        ],
        "memcache.local": "\\OC\\Memcache\\APCu",
        "skeletondirectory": "",
        "updater.release.channel": "stable"
    }
}

Are you using external storage, if yes which one: no

Are you using encryption: no

Server log (data/nextcloud.log)

I see the following error log for each non-watermarked file:

[PHP] Fehler: Error: shell_exec(): Unable to execute 'sh -c 'python /home/www/nextcloud_workflow_scripts/watermark.py --event_type '\''OCP\SystemTag\ISystemTagObjectMapper::assignTags'\'' --file_id '\''2925'\'' --actor_user_id '\''username'\'' --owner_user_id '\''username'\'' --nextcloud_relative_path '\''__groupfolders/1/username/files/path_to_file.pdf'\'' --locally_available_file '\''/home/www/nextcloud/data/__groupfolders/1/path_to_file.pdf'\'' --old_nextcloud_relative_file_path '\'''\''' >/dev/null &' at /home/www/nextcloud/apps/workflow_script/lib/BackgroundJobs/Launcher.php#68 at <<closure>>

0. <<closure>>
   OC\Log\ErrorHandler::onError(2, "shell_exec(): U ... '", "/home/www/nextc ... p", 68, [["python /home/ ... "])
1. /home/www/nextcloud/apps/workflow_script/lib/BackgroundJobs/Launcher.php line 68
   shell_exec("sh -c 'python / ... &")
2. /home/www/nextcloud/lib/public/BackgroundJob/Job.php line 78
   OCA\WorkflowScript\BackgroundJobs\Launcher->run(["python /home/w ... "])
3. /home/www/nextcloud/lib/public/BackgroundJob/QueuedJob.php line 58
   OCP\BackgroundJob\Job->start(["OC\\BackgroundJob\\JobList"])
4. /home/www/nextcloud/lib/public/BackgroundJob/QueuedJob.php line 48
   OCP\BackgroundJob\QueuedJob->start(["OC\\BackgroundJob\\JobList"])
5. /home/www/nextcloud/cron.php line 152
   OCP\BackgroundJob\QueuedJob->execute(["OC\\BackgroundJob\\JobList"], ["OC\\Log"])

at 2023-08-06T16:08:03+00:00

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at apps/workflow_script/lib/BackgroundJobs/Launcher.php line 68 and follow its invocation from cron.php. Reproduce the workflow with multiple tag changes while collecting the reported shell_exec errors and background-job logs. Done means identifying whether the launcher or job concurrency is responsible and documenting or testing the resulting resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, python, shell
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.