My Jetpack: add A/B experiment to welcome flow

[IanRamosC]

Context: pbxNRc-3UJ-p2

Proposed changes:

• This PR adds an A/B experiment to new welcome flow. In order to load the assignment right after site-connecting asynchronously, a new function was added to load w.js synchronously, so it initializes the cookie necessary to load the assignment. This function was added to side-load-tracks.ts.
  For more context about the experiment, see the P2 post linked above.

Other information:

• Have you written new tests for your changes, if applicable?
• Have you checked the E2E test CI results, and verified that your changes do not break them?
• Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

pbxNRc-3UJ-p2

Does this pull request change what data or activity we track or use?

No

Testing instructions:

• Spin up a JN site with Jetpack Beta pointing to this branch and with sandbox access enabled
• Visit your site, go to Settings -> Jetpack Constants on /wp-admin/options-general.php?page=companion_settings and point JETPACK__SANDBOX_DOMAIN to your WPCOM sandbox
• Go to My Jetpack and open your browser's developer console on the network tab
• Click on "Activate Jetpack in one click"

• It'll then redirect you to the user-connection screen
• Filter your network request by assignment and confirm you see a response like this:

{
    "variations": {
        "jetpack_my_jetpack_post_connection_flow_202408": null
    },
    "ttl": 60,
    "debug": {
        "backend_aa_result": "request not sampled"
    }
}

• To test the treatment variation, disconnect your site by going back to My Jetpack, scrolling down to "Connection", clicking on "Manage" and then disconnecting; [new] then go to "Dashboard", scroll down and click "Reset Options (dev only)" in the footer.

• Then, follow the instructions on this page 21912-explat-experiment, hovering "Bookmarklet" for treatment in the Audience section

[github-actions]

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

• To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the add/ab-experiment-welcome-flow branch.

• To test on Simple, run the following command on your sandbox:

  bin/jetpack-downloader test jetpack add/ab-experiment-welcome-flow
  

Interested in more tips and information?

• In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
• Read more about our development workflow here: PCYsg-eg0-p2
• Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

[github-actions]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

• ✅ Include a description of your PR changes.
  
• ✅ Add a "[Status]" label (In Progress, Needs Team Review, ...).
  
• ✅ Add testing instructions.
  
• ✅ Specify whether this PR includes any changes to data or privacy.
  
• ✅ Add changelog entries to affected projects
  

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

---

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

---

Once your PR is ready for review, check one last time that all required checks appearing at the bottom of this PR are passing or skipped.
Then, add the "[Status] Needs Team Review" label and ask someone from your team review the code. Once reviewed, it can then be merged.
If you need an extra review from someone familiar with the codebase, you can update the labels from "[Status] Needs Team Review" to "[Status] Needs Review", and in that case Jetpack Approvers will do a final review of your PR.

[robertsreberski]

I've gone ahead and made some adjustments, but in general code looks good, imo ready to ship!

[robertsreberski]

Merged and resolved conflicts after Tracks PR merge (#38641)

[jboland88]

Code wise this looks good - clever solution to sideload tracks after the site connection.
The test worked as described - nice work.

The main question I had was whether we should send the user straight into the user connection flow for control. Not a blocker though.

[jboland88]

Changes LGTM, tested according to plan.

I tested without sandboxing the API to simulate a "broken" request where no assignments were received and it defaulted to control as expected.