Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Sandbox configuration to more easily perform Integration tests. #533

Merged
merged 10 commits into from
Jul 4, 2024

Conversation

bevanweiss
Copy link
Contributor

@bevanweiss bevanweiss commented Jun 20, 2024

Will automatically 'install' dotnet/dotnet sdk into sandbox at startup based on versions in 'src/test/sandbox/{AMD64,ARM64}' (dotnet-runtime.zip & dotnet-sdk.zip).
Running ./src/test/sandbox/setup_sandbox.bat will download the required files (when run on the host).
If the files aren't available, and the guest has network, it will attempt to download the files itself at startup.

Still to be done (possible future enhancements):

  • menu at startup to allow easy one button selection of which particular test set to run
  • investigation into whether vstest.console.exe can be used to capture test execution, and possibly have the sandbox available as a test runner, so MSI tests could be run in the sandbox directly from the dev environment.

Will automatically 'install' dotnet/dotnet sdk into sandbox at startup
based on versions in 'src/test/sandbox/{AMD64,ARM64}' (dotnet-runtime.zip
& dotnet-sdk.zip).
Running ./src/test/sandbox/setup_sandbox.bat will download the required
files (when run on the host).
If the files aren't available, and the guest has network, it will attempt
to download the files itself at startup.

Still to be done:
* menu at startup to allow easy one button selection of which particular
test set to run
* investigation into whether vstest.console.exe can be used to capture test
execution, and possibly have the sandbox available as a test runner, so MSI
tests could be run in the sandbox directly from the dev environment.


Signed-off-by: Bevan Weiss <[email protected]>
It will now prompt for configuring the Remote Debugger if it can find it
on the host when doing the setup.
And will prompt within the Sandbox to run the Debugger at startup if found
It will also show the host-local IP address to connect to the debugger.

In addition, it will display a small menu to allow for simple selection of
a particular runtests.cmd version to execute.

I still haven't found a good way to get a debugger attached into the msiexec
environment without already having a coded breakpoint in, however.

Also added this sandbox folder to .gitignore, so that artifacts from here
aren't committed into the repo


Signed-off-by: Bevan Weiss <[email protected]>
And a fix up for the tests not launching the first try
(due to delayed expansion).
Also fixed up the copying of the debugger files.
… them.

By default enable /noauth and /anyuser on the debugger, and don't show the
security warning (/nosecuritywarn).
It'll still show the firewall warning prompt.  Default settings should be
fine for firewall (Private networks).

Signed-off-by: Bevan Weiss <[email protected]>
Not sure if it's ideal, but it works..

Signed-off-by: Bevan Weiss <[email protected]>

pushd "%TEMP%"

mkdir "%ProgramFiles%\dotnet"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should prefer using the official installers as more typical of what users will be running. Here's what I did: https://github.com/barnson/wixsandbox

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach of copy'ing the zip in, and unzipping actually came from the Microsoft docker images for containers
https://github.com/dotnet/dotnet-docker/blob/f4904f177893776b719998a8d3a510e55a897b11/src/sdk/8.0/windowsservercore-ltsc2022/amd64/Dockerfile#L46
It's how they do it for the runtime/aspnet/SDK containers.

I did have a quick try (earlier) with the installer, and it seemed to take a bit longer.
It's likely not a significant factor in the overall timing for Sandbox activities though.

I'll modify it tonight to do the .exe download in the host prep, and then the install instead of unzipping within the Sandbox.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just done a comparison on the timing. And it's pretty significant.
Doing the zip file extract, it takes approximately 20 seconds (a little under) from double click to the menu appearing.
Doing the "dotnet.exe /install /quiet /norestart" instead takes over 50 seconds.

Do we really need to go the exe install option?

I actually went and looked at how the https://learn.microsoft.com/en-us/dotnet/core/install/windows#install-the-sdk-1 powershell script does it (it's also downloading the zip and extracting it).
From that I realised I shouldn't have been copying both the runtime AND the SDK over, only the SDK is required... which shaved off about 5 seconds from the startup time. So it was at menu in about 15 seconds.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made another branch in my fork with the EXE version
https://github.com/bevanweiss/wix/tree/sandbox_setup_with_exe
You could do a time trial between this version and the latest zip version if you wanted.
https://github.com/bevanweiss/wix/tree/sandbox_setup

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm less concerned about the time---the tests take ages more---than I am running in a typical environment. (The .zips are for Nano Server, which doesn't support MSI.) How about this? If you have the bundle installers working, leave them in but commented/goto'd out so it's easy enough for someone to switch to them if they want to investigate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the IntegrationMsi tests I was getting about 6 of the Firewall tests failing, with a difference in upper vs lower case of the drive ('C' vs 'c') in the pathname compare.
I haven't checked the IntegrationBurn tests though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm...I'm on Windows 11/22H2. I wonder what would cause that failure.

Copy link
Contributor Author

@bevanweiss bevanweiss Jun 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also on W11 Pro/22H2 10.0.22621
When I run:
fsutil.exe file queryCaseSensitiveInfo
over the various wix folders, it reports:
Case sensitive attribute on directory ..{redacted}\wix\src is disabled.

I'll try going with v6 dotnet for a run, to see if that changes it. I wouldn't think so, but not sure what else might make it different. EDIT: Nope, v6.0.31 made no difference. Still has lower case 'c', whilst upper case 'C' expected

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One difference is that your tests are coming from the Git checkout -- I wonder if Git is doing something weird...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, issue found.
To identify the tests to present (in the menu) I had the following:

for /f %%i in ('where /R c:\build runtests.cmd') do (
...

NOTE: Lowercase c:\, it would store these away, and present them for execution.
If selected for execution it would do a START "Title" /D {PATH} {FILENAME}
Because the where had c:\ then all the paths were c:\... which appears to have influenced the firewall rules created within the MSI run from that path.

src/test/sandbox/TestSandbox.wsb Show resolved Hide resolved
Fix a couple of copy-paste errors in the "don't have files from
host in Sandbox" path also.

Signed-off-by: Bevan Weiss <[email protected]>
Removed ARM64/AMD64 folders for dotnet, the Sandbox MUST match the host
here, so no point downloading the installer for the non-host platform.

Debugger could probably be reduced in the same manner, but have not in
this iteration.
Have also added a simple README, just to explain (very basic) operation.

Signed-off-by: Bevan Weiss <[email protected]>
Install both the x64 AND the x86 versions of Windows Desktop Runtime.
Needed for the Burn Integration tests.

And having dotnet files in the sandbox directory was annoying, so put
them all into an assets directory

Signed-off-by: Bevan Weiss <[email protected]>
Neither "reg import" nor "reg add" appeared to work inside the sandbox
So went with "regedit /s {path}" which did work.

Signed-off-by: Bevan Weiss <[email protected]>
@barnson barnson enabled auto-merge (rebase) July 4, 2024 22:40
@barnson
Copy link
Member

barnson commented Jul 4, 2024

Clean build---thanks!

@barnson barnson merged commit 54df047 into wixtoolset:main Jul 4, 2024
2 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Jul 4, 2024
@bevanweiss bevanweiss deleted the sandbox_setup branch July 17, 2024 10:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants