Improved support for RMarkdown sites

[marcosnav]

Intent

Update
The direction of this PR changed, it isn't automatically including the files and assets that could be required by the project. This change of direction after discussing with the team and surfacing previous feedback from user testing in which users might prefer to add things manually, instead of including files that could represent a security risk, (e.g: pushing secrets unintentionally)

---

Don't be alarmed:
From the 53 files shown in this PR 98% are artifacts used for testing, no need to code review those under .../inspect/detectors/testdata/*.

This PR introduces some changes to improve the experience of deploying RMD sites.

Fixes #1643

Type of Change

• • Bug Fix
• • New Feature
• • Breaking Change
• • Documentation
• • Refactor
• • Tooling

Approach

Connect requires the manifest.json file to specify content_category as site when the document is going to be a website, the common cases for this are projects using _site.yml configurations or _bookdown.yml projects.

The approach in this PR when there is a _site.yml or _bookdown.yml file present:

1. Pick all the existing files and directories located at the level of the chosen entrypoint. This way, the user has a much better head start.
2. The generated manifest.json for the deployment bundle includes the category of site.

User Impact

Deployment of RMD sites should have a much better out of the box experience.

Automated Tests

Added and updated tests for the generated configuration and it's files, and for the update of the manifest file for the content category.

Directions for Reviewers

Update
As mentioned above, with the change of direction in this PR, assets and files will need to be added manually after creating the deployment.

Aside from possible local renv requirements, the following examples can be used to try this out, sites should be easily deployed without much intervention after creating the deployment with Publisher:

• https://github.com/rstudio/connect-content/tree/main/bundles/rmd-site
  • Links should work
• Bookdown demo: https://github.com/rstudio/bookdown-demo
  • Links and assets should look good
  • Important note: By default the bookdown demo has a configuration block to output PDF, handling xelatex system dependencies is not worth the work to test this, plain GitBook or HTML is enough. Updating the _output.yml to the block of content shown below does the trick.

bookdown::gitbook:
  css: style.css
  config:
    toc:
      before: |
        <li><a href="./">A Minimal Book Example</a></li>
      after: |
        <li><a href="https://github.com/rstudio/bookdown" target="blank">Published with bookdown</a></li>
    edit: https://github.com/rstudio/bookdown-demo/edit/master/%s

Checklist

• I have updated CHANGELOG.md to cover notable changes.

[sagerb]

Code looks great, but I raised a few questions. I haven't tried to verify the functionality yet, that is my next step (and I'll creae a new comment with the result).

[sagerb]

I was able to verify the functionality for the rmd_site:

• https://github.com/rstudio/connect-content/tree/main/bundles/rmd-site
  • Links should work

I'm having problems getting the bookdown demo to run. Having problems installing packages within the directories of that repo, almost as if my GITHUB_PAT was invalid. I even went and generated a new one, set it up in my environment and made sure the R sessions were started from terminals with that change to my .zshenv file active. But still no success.

I'm also guessing a lot as to what would be the entrypoint for many of these content types... In the case of rmd_site I picked index.Rmd, and then picked the rmd content type. I believe that is correct?

I'm unsure if my experience with bookdown is a user error or an issue with my configuration. I was able to render the bookdown site, with the changes you specified for _output.yml, but trying to get renv happy with package dependencies set, is basically where I think I got hung up. Any help you can provide in that area, per how you set it up will allow me to verify that functionality as well faster. Thanks!

[marcosnav]

but trying to get renv happy with package dependencies set, is basically where I think I got hung up. Any help you can provide in that area, per how you set it up will allow me to verify that functionality as well faster. Thanks!

@sagerb In that area I only run renv::snapshot in the workspace to ensure everything was lined up. I didn't had to do anything other than that and the update to _output.yml. In case it helps, I'm using R 4.3.1.

[sagerb]

Code changes looked good, thanks!

Further questions on the exclusions, but we're close... :-)

[dotNomad]

The RMD site worked beautifully for me. I did have to select a .Rmd file for the inspection to work. Selecting _site.yml created an unknown type configuration with no included files. That is acceptable, perhaps a follow-up would be to allow that file to work for inspection just to cover all our bases.

Once I figured out some weirdness with renv on my system installing bookdown I was able to deploy with no issues. Similar to the comment above about using _site.yml as the entrypoint, using _bookdown.yml gives type: 'unknown' and could be a nice follow-up issue. Using a Rmd works perfectly.

I'll need to update the CHANGELOG in #2472 as reading the description gave me a different impression on how this worked. I'll get that updated before I mark it as ready to review.

---

Including all but a few files for sites makes sense to me, but understandably we got some push back when we initially had files = ['*'] in the default configuration due to security concerns / accidentally deploying certain files.

The logic here includes a .env file for example which could contain environment variables in a pretty common use case.

We should consider this carefully, and perhaps tighten up the file inclusion to only match certain file extensions.

The generated files is the only bit of this PR I'm concerned with. The thorough testing you have done here makes it rock solid deployment-wise well done 👍

[marcosnav]

We should consider this carefully, and perhaps tighten up the file inclusion to only match certain file extensions.

Looking forward to chat about this in our standup, meanwhile...

Selecting _site.yml created an unknown type configuration with no included files.

That's a good point! Specially when it is possible to select _quarto.yml for quarto websites, it is a small change so I'll add the check to proceed with a site type when the chosen entrypoint is a site configuration file.

[marcosnav]

Pushed a new commit so that a configuration for R Markdown Site projects is created when picking _site.yml or _bookdown.yml files.

[marcosnav]

@dotNomad @sagerb PR ready for another look, removed the logic that added files and directories and scoped the files discovery to only the entrypoint and the site configuration .yml.

[dotNomad]

Fantastic PR. Thank you very much @marcosnav for the changes, discussions, and the thorough testing. I tested with the two examples (rmd-site and the Bookdown demo) and both worked beautifully after the few changes you made.