nextcloud / nextcloud/server

Special characters in filenames

Open
#7,762 58 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

1. to develop feature: files feature: filesystem hotspot: filename handling needs review technical debt
Dominant language
PHP
Stars
36.9k
Forks
5.2k
Avg merge
2d 3h
Merged PRs (30d)
713

Description

Steps to reproduce
  1. Put some files (not all with those characters) in any folder
  2. Include this folder by external storage (local)
Expected behaviour

Every file in this folder shoud be scanned and shown in the files-app.

Actual behaviour

These files came through download on the harddisk of my homeserver. The folder containing the downloaded files are configured as “local” external storage in my nextcloud.
Files and folders with german “umlaute” created by nextcloud in the files-app appear in the file listings. Other files and folders (from download) are ignored by the occ-file-scan.

While file-scan in debug mode the following messages appear in nextcloud.log.
There have to be Lügen instead of L\u00fcgen and Hölle instead of H\u00f6lle for example.

{“reqId”:“X7LIb2Ci8jdOOkqp3leZ”,“level”:0,“time”:“2017-12-26T17:25:29+01:00”,“remoteAddr”:"",“user”:"–",“app”:“OC\Files\Cache\Scanner”,“method”:"–",“url”:"–",“message”:"!!! Path ‘Serien/Zoo/S02E06.Sex, L\u00fcgen und Quallen.mp4’ is not accessible or present !!!",“userAgent”:"–",“version”:“12.0.4.3”}
{“reqId”:“X7LIb2Ci8jdOOkqp3leZ”,“level”:0,“time”:“2017-12-26T17:25:29+01:00”,“remoteAddr”:"",“user”:"–",“app”:“OC\Files\Cache\Scanner”,“method”:"–",“url”:"–",“message”:"!!! Path ‘Serien/Zoo/S02E10.H\u00f6lle in Helsinki.mp4’ is not accessible or present !!!",“userAgent”:"–",“version”:“12.0.4.3”}
Server configuration

Operating system: Ubuntu Server 17.10

Web server: Apache 2.4.27

Database: MySQL

PHP version: PHP 7.1.11-0ubuntu0.17.10.1

Nextcloud version: 12.0.4

Updated from an older Nextcloud/ownCloud or fresh install: fresh install

Where did you install Nextcloud from: nextcloud.com

List of activated apps:

App list
Enabled:
  - dav: 1.3.0
  - federatedfilesharing: 1.2.0
  - files: 1.7.2
  - files_external: 1.3.0
  - files_sharing: 1.4.0
  - files_videoplayer: 1.1.0
  - lookup_server_connector: 1.0.0
  - notifications: 2.0.0
  - oauth2: 1.0.5
  - provisioning_api: 1.2.0
  - theming: 1.3.0
  - twofactor_backupcodes: 1.1.1
  - updatenotification: 1.2.0
  - workflowengine: 1.2.0
Disabled:
  - activity
  - admin_audit
  - comments
  - encryption
  - federation
  - files_pdfviewer
  - files_texteditor
  - files_trashbin
  - files_versions
  - firstrunwizard
  - gallery
  - logreader
  - nextcloud_announcements
  - password_policy
  - serverinfo
  - sharebymail
  - survey_client
  - systemtags
  - user_external
  - user_ldap

Nextcloud configuration:

Config report
{
    "system": {
        "instanceid": "oc65jgv8zf6o",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "toothless.goip.de",
            "toothless.fritz.box"
        ],
        "datadirectory": "\/var\/www\/nextcloud\/data",
        "overwrite.cli.url": "https:\/\/toothless.goip.de",
        "dbtype": "mysql",
        "version": "12.0.4.3",
        "dbname": "nextcloud",
        "dbhost": "localhost",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "skeletondirectory": "",
        "logtimezone": "Europe\/Berlin",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "localhost",
            "port": "6379"
        },
        "htaccess.RewriteBase": "\/",
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_from_address": "jan.noormann",
        "mail_domain": "gmail.com",
        "mail_smtphost": "smtp.gmail.com",
        "mail_smtpport": "587",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "tls"
    }
}

Are you using external storage, if yes which one: local

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Opera, Chrome, Firefox

Operating system: Windows 10

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the occ file-scan entry point and the OC\Files\Cache\Scanner messages shown in nextcloud.log. Reproduce the scan against local external storage using filenames containing German umlauts, then verify that every file and folder appears in the files-app listing.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.