Allow combined UTF-8 characters for filenames on Nextcloud External local storage files
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
On Nextcloud v 27.1.0 I have an external storage of type local.
Having file names with utf-8 character encoding and combined character with combining diaeresis fails to be shown.
The error is shown like the following:
php occ files:scan -p /myPath/myExternalStorageFolder
Starting scan for user 1 out of 1 (me)
Entry "klaus/Hans hätte besser aufgepasst.mp4" will not be accessible due to incompatible encoding
.
.
For example the letter 'ä' can be represented in two utf-8 complaint forms:
- (U+00E4), hexadecimal representation: c3a4
- or a combined form of the character "a" (U+0061) followed by combining diaeresis (U+0308), which is used to add two dots above a character. Hexadecimal representation: 61cc88.
Using the latter method for filenames results in an error due to "wrong" character encoding, even though it is valid UTF-8 character encoding.
Consider a scenario where you have a Spanish keyboard as a German tourist in Mallorca. You may produce a combined UTF-8 character for the letter 'ä' using such a keyboard to name your holiday videos and store them on your memory stick right there.
This report highlights the improvement request.
a workaround:
A tool called 'convmv' is available for Linux and can be installed with the following command: sudo apt install convmv
To correct filenames on your holiday memory stick, following my example, use the following command for a dry run:
convmv -r -f utf-8 -t utf-8 --nfc ./myPhantisticHolydayMemories/*
This command will show what files would be renamed without actually renaming them. After inspecting the actions it would take, you can perform the renaming with the following command:
convmv -r -f utf-8 -t utf-8 --nfc --notest ./myPhantisticHolydayMemories/*
Once you have completed this process, trigger a scan again using:
php occ files:scan -p /myPath/myExternalStorageFolder
You should no longer encounter any "incompatible" encoding errors.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with a decomposed UTF-8 filename in an external local storage and run php occ files:scan -p /myPath/myExternalStorageFolder. Trace the external-storage scan path that reports the filename as having incompatible encoding. Done when valid combined UTF-8 filenames are accepted and shown without that error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100