Skip to content

clone: don't decode URL percent encodings#5187

Merged
ethomson merged 5 commits intolibgit2:masterfrom
ianhattendorf:fix/clone-whitespace
Aug 14, 2019
Merged

clone: don't decode URL percent encodings#5187
ethomson merged 5 commits intolibgit2:masterfrom
ianhattendorf:fix/clone-whitespace

Conversation

@ianhattendorf
Copy link
Contributor

Test fails on at least macOS and Linux with error -1 - unexpected HTTP status code: 400. libgit2 attempts to clone https://dev.azure.com/ianhattendorf/With Space/_git/With Space, due to git_net_url_parse setting URL path with git_buf_decode_percent.

@ianhattendorf ianhattendorf changed the title [WIP] clone: don't decode URL percent encodings clone: don't decode URL percent encodings Jul 24, 2019
@ethomson
Copy link
Member

Yeah, this seems right to me. Looks like this got introduced in #4563, but I agree that we should round-trip path unaltered.

The test is a bit tricky, though. We probably should have a repo for the project in a hosting provider that allows spaces in URLs (which I think is only Azure Repos), so that we're not reliant on your test project.

I can set that up (we already have dev.azure.com/libgit2 for our builds) 🔜 .

@ianhattendorf
Copy link
Contributor Author

Yeah, this seems right to me. Looks like this got introduced in #4563, but I agree that we should round-trip path unaltered.

The test is a bit tricky, though. We probably should have a repo for the project in a hosting provider that allows spaces in URLs (which I think is only Azure Repos), so that we're not reliant on your test project.

I can set that up (we already have dev.azure.com/libgit2 for our builds) 🔜 .

Yep, my test repo was definitely intended to be temporary while I was debugging the issue.

I also noticed we're decoding host, query, and user/pass in git_net_url_parse, however I haven't touched them as it looks like at least user/pass decoding is needed due to redirect issues. I can clone correctly from Azure with a password containing both @ and %20, so I'm assuming the transport handles encoding that portion of the url itself?

Looking at #4563, it looks like there were some issues with SSH and encoding/decoding previously. I can clone correctly via SSH with this fix in place, however just wanted to bring that up.

@ethomson
Copy link
Member

@ianhattendorf I set up https://libgit2@dev.azure.com/libgit2/test/_git/spaces%20in%20the%20name which is a mirror of the existing test repo (https://github.com/libgit2/TestGitRepository), would you mind pointing this test there? Thanks!

@ianhattendorf
Copy link
Contributor Author

@ethomson Sure, I've updated the tests. Would it be possible to get SSH working on the spaces in the name repo? test_online_clone__path_whitespace_ssh passes for me if I change the URL to ssh://git@ssh.dev.azure.com/v3/ianhattendorf/With%20Space/With%20Space and set the GITTEST_REMOTE_SSH_{PUBKEY/KEY} env vars to my key.

Also, I'm confused why it's only failing on Linux. Did I miss a step in adding an online ssh test that resulted in it being skipped on macOS/Windows?

@ianhattendorf ianhattendorf marked this pull request as ready for review August 12, 2019 17:45
@ethomson
Copy link
Member

Would it be possible to get SSH working on the spaces in the name repo?

Hmm, in a perfect world, we would do this indeed. But that would mean setting up an SSH public key in an actual user account in Azure DevOps. Even if that user was set to read-only for repositories, I'm a little bit loathe to set that up without thinking through it more deeply.

Can you just update this branch to include the HTTP tests, and then open a new pull request for the SSH changes? That way we can merge this 🔜 to fix the bug, and then we can revisit the SSH test for completeness as time allows?

Also, I'm confused why it's only failing on Linux. Did I miss a step in adding an online ssh test that resulted in it being skipped on macOS/Windows?

I don't think that we build with libssh2 support on those platforms.

Will add later when infrastructure is configured
@ianhattendorf
Copy link
Contributor Author

Sounds good, I've removed the ssh test and I'll open a PR later to add it back in.

@ethomson
Copy link
Member

Nice one, thanks for the fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants