Boost: add cache rebuild functionality

[donnchawp]

This PR adds the cache rebuild functionality from WP Super Cache to the cache in Boost. Instead of deleting expired files, they will be renamed, and if that page is requested, the cache file will be renamed back to its original name. The first visitor won't use that cache file, they'll generate a new one that overwrites the old one. Other visitors seeing the page will use that old cache page, until the first visitor generates a cache page.

There's a longer explanation in the issue #37054.

This PR differs from the description in the issue in one way. If the "rebuild file" is older than the cache TTL, the cache file is deleted and not used.
If a visitor resets a rebuild file and renames it back to the old cache file, the header sent will change from hit or miss to rebuild.

This feature is enabled by default in WPSC when site owners enable caching on the "easy" settings page, and this will be enabled by default in Boost as well. There is no UI to disable it.

The rename() commands added have to be silenced because two visitors may hit the site at the same time and go through the same code, but one will rename the file before the other.

Should I add a constant to disable rebuild mode?

Fixes #37054

Proposed changes:

• When attempting to serve a cached page, the system will look for a "rebuild cache file", and rename it back to its original name.
• If the file to be renamed is too old, it is deleted.
• The rebuild cache file isn't used by the first visitor
• Anywhere where a cache file is deleted, apart from uninstalling, the file is rebuilt instead.
• The delete_directory function has been renamed walk_directory because it handles rebuilding files too.
• The recursive delete_expired_files function has been renamed to gc_expired_files and has an extra parameter, $action that tells it to delete or rebuild files.
• Added rebuild_file and restore_file functions to handle the renaming.
• Added reset_rebuild_file function to check if rebuild file is there, the age, and renaming it back.
• The garbage_collect function now calls delete_expired_files with the action set to REBUILD.
• The invalidate function was updated to support rebuilding.
• The "action", rebuild or delete, must be passed to various functions because the plugin needs to delete the cache files sometimes, like if a post is deleted.

Other information:

• Have you written new tests for your changes, if applicable?
• Have you checked the E2E test CI results, and verified that your changes do not break them?
• Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

pc9hqz-2QH-p2

Does this pull request change what data or activity we track or use?

no

Testing instructions:

• Install this PR
• Enable the Cache module and enable logging.
• As a logged out visitor, go to one of the blog posts to generate a cache file.
• Bring up the webdev network console, and filter out everything but html. Confirm you can see the "X-Jetpack-Boost-Cache: hit" header.
• Edit that page in another window and update it.
• Reload the page in the logged out window and verify the "X-Jetpack-Boost-Cache" header value is now "rebuild".
• You can also check the cache directory after you've edited the page as the file should have the extension ".rebuild.html". After viewing the page in a logged out window, the file will be updated with the new cache file.
• You could probably test with multiple clients by using a shell script to launch multiple wget or curl commands with "&" to put them in the background.
• Make a post private, and make sure the post can't be seen by a logged out browser.
• Delete a comment and make sure the post cache file is deleted.

[github-actions]

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

• ✅ Include a description of your PR changes.
  
• ✅ Add a "[Status]" label (In Progress, Needs Team Review, ...).
  
• ✅ Add testing instructions.
  
• ✅ Specify whether this PR includes any changes to data or privacy.
  
• ✅ Add changelog entries to affected projects
  

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

---

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

---

Once your PR is ready for review, check one last time that all required checks appearing at the bottom of this PR are passing or skipped.
Then, add the "[Status] Needs Team Review" label and ask someone from your team review the code. Once reviewed, it can then be merged.
If you need an extra review from someone familiar with the codebase, you can update the labels from "[Status] Needs Team Review" to "[Status] Needs Review", and in that case Jetpack Approvers will do a final review of your PR.

---

Jetpack plugin:

The Jetpack plugin has different release cadences depending on the platform:

• WordPress.com Simple releases happen daily.
• WoA releases happen weekly.
• Releases to self-hosted sites happen monthly. The next release is scheduled for June 4, 2024 (scheduled code freeze on June 3, 2024).

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

---

Boost plugin:

• Next scheduled release: June 4, 2024.
• Scheduled code freeze: May 27, 2024.

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

[github-actions]

Are you an Automattician? Please test your changes on all WordPress.com environments to help mitigate accidental explosions.

• To test on WoA, go to the Plugins menu on a WordPress.com Simple site. Click on the "Upload" button and follow the upgrade flow to be able to upload, install, and activate the Jetpack Beta plugin. Once the plugin is active, go to Jetpack > Jetpack Beta, select your plugin, and enable the add/boost/cache-rebuild branch.

• To test on Simple, run the following command on your sandbox:

  bin/jetpack-downloader test jetpack add/boost/cache-rebuild
  

Interested in more tips and information?

• In your local development environment, use the jetpack rsync command to sync your changes to a WoA dev blog.
• Read more about our development workflow here: PCYsg-eg0-p2
• Figure out when your changes will be shipped to customers here: PCYsg-eg5-p2

[haqadn]

I have proposed some suggestions/questions. Let me know what you think.

[haqadn]

DELETE_ALL and DELETE_FILE appear to do the same thing. The same is true for REBUILD_ALL and REBUILD_FILE. Any chance we can combine each pair?

[donnchawp]

FILE was supposed to be apply to a single file, but then got applied to a directory, which is what DELETE_ALL does. DELETE_FILE isn't used anywhere either, so these can definitely be simplified.

[donnchawp]

I looked again at the code, and we need to keep DELETE_FILE. It's used when someone leaves a comment, and it's moderated, AND they clicked the "remember me" checkbox. We only want to delete the cache file for them, and nobody else. That bit of code used REBUILD_FILE, but since this file is unique to this user, there's no point rebuilding it. Nobody else will use it, so I replaced it with DELETE_FILE.

While it isn't used now, I'd like to keep REBUILD_FILE too, as it may well be used in the future.

[haqadn]

I've added a couple of suggestion. However, I tried to test this and faced some issues:
The rebuild doesn't work. Ones a page is updated, the next load is a miss. I think this is happening because the file names aren't being compared correctly. Here is what I found in the directory:

index.html
68e7bda3bf0540bbcb4eaed73e14e98a.html
68e7bda3bf0540bbcb4eaed73e14e98a.html.rebuild.html.rebuild.html

[donnchawp]

@haqadn I figured out why you were getting a cache miss on the rebuild file. The TTL was 10 seconds, but when a file is renamed, the filemtime isn't modified. I'll fix that.

I fixed the rename too so it checks if it's renamed already.

[haqadn]

Needs a trunk merge. But, works otherwise.

[dilirity]

Nice! Followed the instructions and it behaves as expected.