rclone / rclone/rclone

Fix(huaweidrive): Replace fullwidth encoding with percent-encoding (%XX)

Open
#9,410 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
59.9k
Forks
5.4k
Avg merge
2d 14h
Merged PRs (30d)
48

Description

Fix: Replace fullwidth encoding with percent-encoding (%XX)

I've pushed a fix in commit ace14fb85 that resolves the 19 FsEncoding + FsPutFiles CI failures.

Root Cause

The previous scheme substituted restricted characters with their Unicode fullwidth equivalents (e.g., ? U+FF1F, . U+FF0E). This worked on most Huawei Drive deployments, but the CI account's API applies NFKC normalization before filename validation, which converts fullwidth ASCII variants straight back to their ASCII originals:

Stored NFKC → Hits blacklist
U+FF1F ?
U+FF0E . ✓ (whole-name)
U+FF1C <
U+2400 \x00

This triggered HTTP 400 errorCode 21004002 on every retry.

Fix

Replace the opt.Enc-based fullwidth scheme with a custom percent-encoding scheme (hwEncodeFilename / hwDecodeFilename):

  • ?%3F, . (whole-name) → %2E, <%3C, \x00%00, etc.
  • % itself → %25 (to keep encoding invertible)
  • Leading space/dot/tilde and trailing space/dot also percent-encoded
  • Invalid UTF-8 bytes encoded one byte at a time (\xfe%FE)

The % character is not on the Huawei API blacklist, and pure ASCII %XX sequences are completely stable under NFKC normalization.

The encoding config option has been removed — the backend now manages filename encoding internally.

Verification

Local integration test results (all 19 FsEncoding subtests + FsPutFiles):

--- PASS: TestIntegration/FsMkdir/FsEncoding/control_chars
--- PASS: TestIntegration/FsMkdir/FsEncoding/dot
--- PASS: TestIntegration/FsMkdir/FsEncoding/dot_dot
--- PASS: TestIntegration/FsMkdir/FsEncoding/punctuation
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_space
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_tilde
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_CR/LF/HT/VT
--- PASS: TestIntegration/FsMkdir/FsEncoding/leading_dot
--- PASS: TestIntegration/FsMkdir/FsEncoding/trailing_space/CR/LF/HT/VT/dot
--- PASS: TestIntegration/FsMkdir/FsEncoding/invalid_UTF-8
--- PASS: TestIntegration/FsMkdir/FsEncoding/URL_encoding
--- PASS: TestIntegration/FsMkdir/FsPutFiles  (including "hello? sausage" path)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Huawei Drive backend's hwEncodeFilename and hwDecodeFilename entry points, then review the FsEncoding and FsPutFiles integration tests named in the issue. Verify percent-encoding remains invertible across the listed filename cases and that the removed encoding option is no longer needed. Done means all 19 FsEncoding subtests and FsPutFiles pass against Huawei Drive.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.