Add/jetpack search custom results

[robfelty]

We frequently get requests from website owners who would like specific posts to be returned for specific queries. This can be thought of as "pinned posts" or "sponsored posts". This change adds the ability to specify which posts should be put at the top of search results for given queries.

Proposed changes:

Other information:

The queries and matching documents can be specified in PHP, e.g. in a theme's functions.php file like so

add_action( 'init', 'twentytwentyfour_pattern_categories' );

function jps_custom_results( $options ) {
  $custom_results = get_option( 'jetpack_search_custom_results' );
  if ( $custom _results ) {
    $options['customResults'] = $custom_results;
  } else {
    $options['customResults'] = array(
      array(
        'pattern' => 'best WordPress plugins',
        'ids' => array(
            "123-p-8",
            "456-9",
        ),
      ),
      array(
        'pattern' => 'regex:(how )?(to )develop WordPress',
        'ids' => array(
          "123-p-6",
        ),
      ),
    );
  }
    return $options;
}

add_filter( 'jetpack_instant_search_options', 'jps_custom_results' );

Note that the document ids are of the form `<blog_id>-p-<post_id>'. This format is relatively easy to understand, though it does require one to know the blog_id of their site. This can be found by inspecting the queries to the public API when performing a search in the developer tools of a browser. The format is also flexible enough such that if you are searching across multiple blogs simultaneously, it allows one to specify pinned posts from multiple blogs. Also note that the order in which the ids are coded is the order in which the results will be returned. It is possible to specify up to 100 documents IDs for a given query pattern. If you specify more, it will simply be truncated.

There are two types of queries - by default, we will use exact matching. However, it is also possible to specify a query with a regular expression by preceding the query with "regex:". The regexes are evaluated anchored to the beginning and end of the query, such that "regex:hello.world" will match the query string "hello beautiful world", but not "i say hello to the world" or "hello world i love you". If you would like to get rid of this anchoring, you can add "." at the beginning or end of your pattern (or both). This is similar to how regular expressions work in Java. The reasoning behind this is that regular expressions are complicated and it is easy for inexperience developers to get in trouble with them, so we are trying to avoid that as much as possible. Note that the order in which the rules are added is important. The first matching rule will be used.

• Have you written new tests for your changes, if applicable?
• Have you checked the E2E test CI results, and verified that your changes do not break them?
• Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

For now, I think we should only implement this with advanced users in mind, thus requiring PHP code. In the future, we might want to create a UI/UX to allow users to add rules in wp-admin.

We may also want to add a way to mark that these posts are being artificially added to the top of the search results.

Does this pull request change what data or activity we track or use?

Testing instructions:

In order to test this you will need a Jetpack site with Jetpack Search installed and active. In order to activate Jetpack Search you must have a user connection and purchase it. You can purchase the free plan. You will need to make sure that the site is properly synced and that the site has been indexed. You can check this by looking at the Jetpack Debugger or by looking at the Jetpack Search settings page, which will tell you how many documents have been indexed.

You also need to sandbox the public api, and apply this patch on your sandbox: D165510-code
After applying that patch, edit public.api/rest/wpcom-json-endpoints/class.wpcom-json-api-search-site-v1-3-endpoint.php adding a line like this around line 561: Use whatever blog_id you are testing with.

				$all_a8c_p2s[] = 211634998;

We are only allowing the custom results functionality on a limited number of sites right now. That is why you need to add it manually there.

Add several posts of your choosing. I tested by adding a post with the title "hello universe". I then added a rule saying that searching for "hello world" should return the "hello universe" post on top, even though the default "hello world" post would be a better match. I added this in my themes functions.php file like so:

add_action( 'init', 'twentytwentyfour_pattern_categories' );
function jps_custom_results( $options ) {
    $options['customResults'] = array(
        array(
            'pattern' => 'regex:aff.*',
            'ids' => array( "211634998-p-6",)
        ),
        array(
            'pattern' => 'hello world',
            'ids' => array( "211634998-p-6",)
        ),
    );
    return $options;
}
add_filter( 'jetpack_instant_search_options', 'jps_custom_results' );

After adding this to the functions.php file, if you perform a search on your site, you should see the "hello universe" post come out as the top result

[github-actions]

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

• To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the add/jetpack-search-custom-results branch.

• To test on Simple, run the following command on your sandbox:

  bin/jetpack-downloader test jetpack add/jetpack-search-custom-results
  

Interested in more tips and information?

• In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
• Read more about our development workflow here: PCYsg-eg0-p2
• Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

[github-actions]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

• ✅ Include a description of your PR changes.
  
• ✅ Add a "[Status]" label (In Progress, Needs Team Review, ...).
  
• ✅ Add testing instructions.
  
• ✅ Specify whether this PR includes any changes to data or privacy.
  
• ✅ Add changelog entries to affected projects
  

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

---

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

---

Follow this PR Review Process:

1. Ensure all required checks appearing at the bottom of this PR are passing.
2. Choose a review path based on your changes:
   • A. Team Review: add the "[Status] Needs Team Review" label
     • For most changes, including minor cross-team impacts.
     • Example: Updating a team-specific component or a small change to a shared library.
   • B. Crew Review: add the "[Status] Needs Review" label
     • For significant changes to core functionality.
     • Example: Major updates to a shared library or complex features.
   • C. Both: Start with Team, then request Crew
     • For complex changes or when you need extra confidence.
     • Example: Refactor affecting multiple systems.
3. Get at least one approval before merging.

Still unsure? Reach out in #jetpack-developers for guidance!

---

Jetpack plugin:

The Jetpack plugin has different release cadences depending on the platform:

• WordPress.com Simple releases happen semi-continuously (PCYsg-Jjm-p2).
• WoA releases happen weekly.
• Releases to self-hosted sites happen monthly. The next release is scheduled for none scheduled (scheduled code freeze on undefined).

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

---

Search plugin:

• Next scheduled release: none scheduled.

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

[gibrown]

I wonder if we should get some feedback from Crew on this diff in general. I feel like there may be ways to improve the UX of this filter. For instance, could we have a call something like:

jps_add_custom_result( 'best WordPress plugins', $blog_id, $post_id );

Maybe we don't want that due to wanting to support both posts and comments, but it feels like it would be good to make it more readable and allowing easily reviewing a potentially long set of customizations.

All of these customizations will also show up in the page source, which is probably fine, but feels a bit weird.

[robfelty]

I wonder if we should get some feedback from Crew on this diff in general. I feel like there may be ways to improve the UX of this filter. For instance, could we have a call something like:

jps_add_custom_result( 'best WordPress plugins', $blog_id, $post_id );

Maybe we don't want that due to wanting to support both posts and comments, but it feels like it would be good to make it more readable and allowing easily reviewing a potentially long set of customizations.

All of these customizations will also show up in the page source, which is probably fine, but feels a bit weird.

I see your point, but I also feel like this is something we could add later. I developed this feature in this way, mimicking the adminQueryFilter option, which is quite similar (and also ends up in the page source). Neither of these have any PHP APIs really, they are both just available filters. Ultimately, I think that if we want to make it easier to customize Jetpack Search, especially with this option, it would be better to create a UI/UX in the settings in wp-admin which would make it easy to add custom rules, and which would not require users to know blogids, postids, or comment ids. That UI could use any number of wrapper functions which would set this option.

It is also possible that there is a better solution going forward than the way that adminQueryFilter is implemented. As far as I can tell though, any sort of PHP filter which we offer has to make its way into the javascript side of things in some way.

[github-actions]

This PR has been marked as stale. This happened because:

• It has been inactive for the past 3 months.
• It hasn’t been labeled `[Pri] BLOCKER`, `[Pri] High`, etc.

If this PR is still useful, please do a [trunk merge or rebase](https://github.com/Automattic/jetpack/blob/trunk/docs/git-workflow.md#keeping-your-branch-up-to-date) and otherwise make sure it's up to date and has clear testing instructions. You may also want to ping possible reviewers in case they've forgotten about it. Please close this PR if you think it's not valid anymore — if you do, please add a brief explanation.

If the PR is not updated (or at least commented on) in another month, it will be automatically closed.

[kangzj]

Thanks for working on this Robert! This is a feature users longing for 👍

It tests well and looks good. It doesn't break without the diff+filters , so I think it's safe to ship. I'd avoid the current release if possible tho.

[robfelty]

@kangzj - thanks for testing. I don't think it is urgent to get into the latest release. What do I need to do to put it into the next release? I would like to merge before it gets stale, but then release whenever makes sense.

[kangzj]

@robfelty