Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enabling Windows tests #376

Merged
merged 10 commits into from
Oct 12, 2023
Merged

Conversation

mjcarroll
Copy link
Contributor

@mjcarroll mjcarroll commented Sep 18, 2023

Fix tests around Windows' handling (or not handling) of : and @ in filesystem paths.

To address this, I did a few things.

  1. If a URI has an authority (that is a username or a port in the format username@host:port), then we URL encode that to get a "safe" path.

  2. Make sure that we are passing hasAuthority = true for all of our URIs that use https. This may be a behavior that we want to change in gz-common eventually, but is sufficient for here.

  3. Fixes UniqueName to generate valid paths for both ModelIdentifier and WorldIdentifier. I also choose to make UniqueName always return unix-delimited paths, which cleans up some windows-specific test logic.

I believe that this resolves the test failures in FuelClientTest.CachedWorld and FuelClientTest.PatchModelFail as described in #373.

When forward ported to main, this should resolve #231

This (finally) resolves #106

@github-actions github-actions bot added the 🎵 harmonic Gazebo Harmonic label Sep 18, 2023
@mjcarroll mjcarroll changed the title Re-enable all Windows tests to see what breaks Re-enabling Windows tests Sep 18, 2023
@mjcarroll mjcarroll self-assigned this Sep 19, 2023
@mjcarroll mjcarroll force-pushed the mjcarroll/fix_windows_colon branch from 7f5c449 to aa8695c Compare September 20, 2023 17:12
Signed-off-by: Michael Carroll <[email protected]>
Signed-off-by: Michael Carroll <[email protected]>
Signed-off-by: Michael Carroll <[email protected]>
Signed-off-by: Michael Carroll <[email protected]>
Signed-off-by: Michael Carroll <[email protected]>
Signed-off-by: Michael Carroll <[email protected]>
@mjcarroll mjcarroll force-pushed the mjcarroll/fix_windows_colon branch from 02a1f33 to 14669c8 Compare September 27, 2023 14:00
Signed-off-by: Michael Carroll <[email protected]>
@mjcarroll mjcarroll force-pushed the mjcarroll/fix_windows_colon branch from 3a9c26a to 839deae Compare September 27, 2023 14:17
@mjcarroll mjcarroll marked this pull request as ready for review September 27, 2023 14:20
@mjcarroll mjcarroll requested a review from nkoenig as a code owner September 27, 2023 14:20
@mjcarroll mjcarroll requested a review from azeey September 27, 2023 15:15
Copy link
Contributor

@azeey azeey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just a couple of questions.

Also, have you seen gazebosim/gz-common#38? I'm wondering if we did proper encoding in gz-common, we wouldn't need some of the changes in this PR.

/// \param[in] _id The model identifier
/// \param[out] _path Local path where the model can be found.
/// \return FETCH_ERROR if not cached, FETCH_ALREADY_EXISTS if cached.
public: Result CachedModel(const ModelIdentifier &_id,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to do this for CacheWorld or is it not necessary since we don't include worlds in SDF files?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't covered by existing tests, so I probably didn't notice it. I suppose it's unnecessary without include-ing in SDF.

@@ -1425,8 +1443,7 @@ Result FuelClient::DownloadModel(const common::URI &_modelUrl,
}

_path = gz::common::joinPaths(this->Config().CacheLocation(),
id.Server().Url().Path().Str(), id.Owner(), "models", id.Name(),
id.VersionStr());
id.UniqueName(), id.VersionStr());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that UniqueName uses Server().Url().Str(), while the code here previously had Server().Url().Path().Str(). Is this intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because previously when we had a web URL, we weren't setting the authority and treating the whole thing as a path, when really the domain name is the authority in this case

# Before (authority = false)
https://fuel.gazebosim.com/user/model
  Authority: ""
  Path: "fuel.gazebosim.com/user/model"

https://foo@localhost:8080/user/model -> 
  Authority: ""
  Path: "foo@localhost:8080/user/model"

# After (authority = true)
https://fuel.gazebosim.com/user/model
  Authority: "fuel.gazebosim.com"
  Path: "user/model"

https://foo@localhost:8080/user/model -> 
  Authority: "foo@localhost:8080"
  Path: "user/model"

So we need the "full" URL returned by Url() to get the Authority as part of the unique name.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I misunderstood Path as a function that somehow converts the URL into a file path, not returning the formal Path of a URL. This makes sense. Thanks!

@mjcarroll
Copy link
Contributor Author

I'm wondering if we did proper encoding in gz-common, we wouldn't need some of the changes in this PR.

I think that's likely the case, but when we discussed it earlier, I think we decided that making that change in gz-common5 would be too much and potentially be an API breakage? This was some time ago, so we may be able to revisit that.

Additionally, with bumping gz-common, it may be that we can just fix it there and most of this goes away in a forward port to main?

@azeey
Copy link
Contributor

azeey commented Oct 12, 2023

Additionally, with bumping gz-common, it may be that we can just fix it there and most of this goes away in a forward port to main?

Okay, I've added to the list of things to consider working on for Ionic. I think this is good to merge!

@Crola1702
Copy link
Contributor

Could this be backported to fix #391?

@mjcarroll
Copy link
Contributor Author

Could this be backported to fix #391?

No, we determined that the behavior change couldn't be ported back, so the tests should be disabled as in #387

@azeey azeey added needs upstream release Blocked by a release of an upstream library and removed needs upstream release Blocked by a release of an upstream library labels Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🎵 harmonic Gazebo Harmonic
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Failing Windows CI tests on main branch Windows doesn't support colons in filenames
3 participants