nextcloud / nextcloud/server

[Bug]: Can't create tag with some Thai characters from Files and Recognize

Open
#41,178 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

0. Needs triage 27-feedback bug feature: language/translations (l10n/i18n) feature: tags
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

⚠️ This issue respects the following points: ⚠️
Bug description

Because problem persists across two (or more apps) and same entry can be created directly using psql, belief is it is at Server/core level.
Problem is of nature that can't create tag with some Thai characters from Files and Recognize .

"Classifier process output: [\n  {\n    className: '\u0e1e\u0e23\u0e30\u0e17\u0e35\u0e48\u0e19\u0e31\u0e48\u0e07\u0e2d\u0e32\u0e20\u0e23\u0e13\u0e4c\u0e20\u0e34\u0e42\u0e21\u0e01\u0e02\u0e4c\u0e1b\u0e23\u0e32\u0e2a\u0e32\u0e17',\n    probability: 0.9035003781318665\n  }\n]\n","userAgent":"--","version":"27.1.2.1","data":{"app":"recognize"}}

Worth mentioning that other tag with Thai letters could be created from Files: อินพุต-NC-level-tagged

version: 27.1.2.1

Steps to reproduce
  1. Create file or folder.
  2. Try to create tag from Files: พระที่นั่งอาภรณ์ภิโมกข์ปราสาท
Expected behavior

Create tag without an error.

Installation method

Docker (Debian)

Nextcloud Server version

27

Operating system

None

PHP engine version

8.2

Web server

Nginx

Database engine version

Postgres

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?
  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other
Configuration report

No response

List of activated Apps

No response

Nextcloud Signing status

No response

Nextcloud Logs
"app":"core","method":"","url":"/cron.php","message":"Error while running background job (class: OCA\\Recognize\\BackgroundJobs\\ClassifyLandmarksJob, arguments: Array\n(\n    [storageId] => 10\n    [rootId] => 31946\n)\n)","userAgent":"--","version":"27.1.2.1","exception":{"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing a query: SQLSTATE[22021]: Character not in repertoire: 7 ERROR:  invalid byte sequence for encoding \"UTF8\": 0xe0\nCONTEXT:  unnamed portal parameter $1","Code":7,"Trace":[{"file":"/var/www/html/3rdparty/doctrine/dbal/src/Connection.php","line":1814,"function":"convert","class":"Doctrine\\DBAL\\Driver\\API\\PostgreSQL\\ExceptionConverter","type":"->","args":[["Doctrine\\DBAL\\Driver\\PDO\\Exception"],["Doctrine\\DBAL\\Query"]]},{"file":"/var/www/html/3rdparty/doctrine/dbal/src/Connection.php","line":1749,"function":"handleDriverException","class":"Doctrine\\DBAL\\Connection","type":"->","args":[["Doctrine\\DBAL\\Driver\\PDO\\Exception"],["Doctrine\\DBAL\\Query"]]},{"file":"/var/www/html/3rdparty/doctrine/dbal/src/Connection.php","line":1055,"function":"convertExceptionDuringQuery","class":"Doctrine\\DBAL\\Connection","type":"->","args":[["Doctrine\\DBAL\\Driver\\PDO\\Exception"],"SELECT * FROM \"oc_systemtag\" WHERE (\"name\" = ?) AND (\"visibility\" = ?) AND (\"editable\" = ?)",[null,1,1],[]]},{"file":"/var/www/html/lib/private/DB/Connection.php","line":262,"function":"executeQuery","class":"Doctrine\\DBAL\\Connection","type":"->","args":["SELECT * FROM \"oc_systemtag\" WHERE (\"name\" = ?) AND (\"visibility\" = ?) AND (\"editable\" = ?)",[null,1,1],[],null]},{"file":"/var/www/html/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":345,"function":"executeQuery","class":"OC\\DB\\Connection","type":"->","args":["SELECT * FROM \"oc_systemtag\" WHERE (\"name\" = :name) AND (\"visibility\" = :visibility) AND (\"editable\" = :editable)",[null,1,1],[]]},{"file":"/var/www/html/lib/private/DB/QueryBuilder/QueryBuilder.php","line":280,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/html/lib/private/SystemTag/SystemTagManager.php","line":186,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/html/custom_apps/recognize/lib/Service/TagManager.php","line":31,"function":"getTag","class":"OC\\SystemTag\\SystemTagManager","type":"->","args":["*** sensitive parameters replaced ***",true,true]},{"file":"/var/www/html/custom_apps/recognize/lib/Service/TagManager.php","line":71,"function":"getTag","class":"OCA\\Recognize\\Service\\TagManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"function":"OCA\\Recognize\\Service\\{closure}","class":"OCA\\Recognize\\Service\\TagManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/custom_apps/recognize/lib/Service/TagManager.php","line":70,"function":"array_map","args":[["Closure"],["landmark","*** sensitive parameters replaced ***"]]},{"file":"/var/www/html/custom_apps/recognize/lib/Classifiers/Images/LandmarksClassifier.php","line":47,"function":"assignTags","class":"OCA\\Recognize\\Service\\TagManager","type":"->","args":[531275,["landmark","*** sensitive parameters replaced ***"]]},{"file":"/var/www/html/custom_apps/recognize/lib/BackgroundJobs/ClassifyLandmarksJob.php","line":41,"function":"classify","class":"OCA\\Recognize\\Classifiers\\Images\\LandmarksClassifier","type":"->","args":[[["OCA\\Recognize\\Db\\QueueFile",343962],["OCA\\Recognize\\Db\\QueueFile",343963],["OCA\\Recognize\\Db\\QueueFile",343964],
[...CUT...]
["OCA\\Recognize\\Db\\QueueFile",346040],["OCA\\Recognize\\Db\\QueueFile",346041],["OCA\\Recognize\\Db\\QueueFile",346042],["OCA\\Recognize\\Db\\QueueFile",346043],["OCA\\Recognize\\Db\\QueueFile",346044]]]},{"file":"/var/www/html/custom_apps/recognize/lib/BackgroundJobs/ClassifierJob.php","line":73,"function":"classify","class":"OCA\\Recognize\\BackgroundJobs\\ClassifyLandmarksJob","type":"->","args":[[["OCA\\Recognize\\Db\\QueueFile",343962],["OCA\\Recognize\\Db\\QueueFile",343963],["OCA\\Recognize

[...cut...]

\\QueueFile",346041],["OCA\\Recognize\\Db\\QueueFile",346042],["OCA\\Recognize\\Db\\QueueFile",346043],["OCA\\Recognize\\Db\\QueueFile",346044]]]},{"file":"/var/www/html/custom_apps/recognize/lib/BackgroundJobs/ClassifyLandmarksJob.php","line":33,"function":"runClassifier","class":"OCA\\Recognize\\BackgroundJobs\\ClassifierJob","type":"->","args":["landmarks",[10,31946]]},{"file":"/var/www/html/lib/public/BackgroundJob/Job.php","line":81,"function":"run","class":"OCA\\Recognize\\BackgroundJobs\\ClassifyLandmarksJob","type":"->","args":[[10,31946]]},{"file":"/var/www/html/lib/public/BackgroundJob/TimedJob.php","line":103,"function":"start","class":"OCP\\BackgroundJob\\Job","type":"->","args":[["OC\\BackgroundJob\\JobList"]]},{"file":"/var/www/html/lib/public/BackgroundJob/TimedJob.php","line":93,"function":"start","class":"OCP\\BackgroundJob\\TimedJob","type":"->","args":[["OC\\BackgroundJob\\JobList"]]},{"file":"/var/www/html/cron.php","line":152,"function":"execute","class":"OCP\\BackgroundJob\\TimedJob","type":"->","args":[["OC\\BackgroundJob\\JobList"],["OC\\Log"]]}],"File":"/var/www/html/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","Line":101}},"message":"Error while running background job (class: OCA\\Recognize\\BackgroundJobs\\ClassifyLandmarksJob, arguments: Array\n(\n    [storageId] => 10\n    [rootId] => 31946\n)\n)","exception":{},"CustomMessage":"Error while running background job (class: OCA\\Recognize\\BackgroundJobs\\ClassifyLandmarksJob, arguments: Array\n(\n    [storageId] => 10\n    [rootId] => 31946\n)\n)"}}


### Additional info

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue by creating the Thai tag from Files, then inspect custom_apps/recognize/lib/Service/TagManager.php and lib/private/SystemTag/SystemTagManager.php around the PostgreSQL query shown in the log. Trace the call from LandmarksClassifier.php and ClassifyLandmarksJob.php; done means the tag can be created and the recognition background job completes without the encoding error.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.