webdav: use URLPathEscapeAll for RFC 3986 compliant path encoding#9180
Open
veeceey wants to merge 1 commit intorclone:masterfrom
Open
webdav: use URLPathEscapeAll for RFC 3986 compliant path encoding#9180veeceey wants to merge 1 commit intorclone:masterfrom
veeceey wants to merge 1 commit intorclone:masterfrom
Conversation
Use URLPathEscapeAll instead of URLPathEscape for path encoding. URLPathEscape relies on Go's url.URL.String() which only minimally escapes paths - reserved sub-delimiter characters like semicolons and equals signs pass through unescaped. Per RFC 3986 section 3.3, these characters must be percent-encoded when used as literal values in path segments. Some WebDAV servers (notably dCache/Jetty) interpret unescaped semicolons as path parameter delimiters, which truncates filenames at the semicolon position. URLPathEscapeAll encodes everything except [A-Za-z0-9/], which is safe for all servers. Fixes rclone#9082
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #9082
The WebDAV backend uses
URLPathEscapefor path encoding, which relies on Go'surl.URL.String(). This only does minimal escaping and lets reserved sub-delimiter characters like semicolons (;) and equals signs (=) through unescaped.Per RFC 3986 section 3.3, these characters must be percent-encoded when used as literal path values. Some WebDAV servers (dCache running on Jetty in the reporter's case) interpret unescaped semicolons as path parameter delimiters per the RFC, which silently truncates the filename at the semicolon position.
Switched to
URLPathEscapeAllwhich encodes everything except[A-Za-z0-9/]. As @ncw mentioned in the issue, the extra encoding overhead is negligible compared to data transfer and round trips.Added a test that spins up a mock WebDAV server and verifies semicolons in filenames are percent-encoded in the request path.