ci: Build the VSCode extension using GitHub Actions

[vogelsgesang]

This commit adds a Github action which builds the VS Code extension. The .vsix file is uploaded as an artifcat such that it can be easily downloaded from GitHub (in the "Artifacts" section of the action's summary page). Those artifacts are also uploaded as part of pull requests, which makes it easier to quickly download, install and test the extension built from a pull request.

In addition, if the workflow is triggered because a new release was tagged, the .vsix is also attached as an artifact to the newly created release.

As part of this change, the vsce tool is now listed as a devDependency and thereby its exact version is now part of the package-lock.json, making the packaging step more reproducible. The new, recommend way to package the extension is now npm run package.

[vogelsgesang]

See https://github.com/vogelsgesang/vscode-bazel/actions/runs/8530517980 for an example run

[vogelsgesang]

Note that this does not conflict with #340. Even if we run the build as part of a Github Action instead of bazelci, we could still call bazel instead of npm run package.

It brings up the question, though, if we want to keep the bazelci integration at all, or if we want to switch over to Github Actions completely.

[cameron-martin]

Building in both github actions and bazelci doesn't seem ideal. I wonder how easy it'd be to use the build artifacts from bazelci?

[vogelsgesang]

Building in both github actions and bazelci doesn't seem ideal.

If you want, I can remove the bazelci config, then we would be 100% on Github

I wonder how easy it'd be to use the build artifacts from bazelci?

I see a couple of challenges with Bazel CI:

• I cannot experiment with the Bazel CI workflows on their private repository. This makes it extremely hard to iterate on the CI definition. Github Actions are much easier to setup and modify / maintain since they also work for forks and changes can therefore first be tested in a personal fork.
• We need some authentication token, otherwise we won't be able to access the Github APIs. Afaict, bazelci offers no GH_TOKEN equivalent?
• Afaict, Bazel CI can only trigger on code changes, but not on "release" events on Github?

What are the benefits of using Bazel CI?

[cameron-martin]

I'm not sure personally. Maybe some of the other maintainers can give some context on why we (plus a lot of the other repos in this org) use bazelci over github actions? cc @coeuvre @jfirebaugh

[vogelsgesang]

Converting to a draft, since I would still like to make some changes, in case we actually decide that we actually want to start using GitHub Actions for building

[meteorcloudy]

Bazel team here, I think Bazel CI provides better integration with Bazel, like specifying build/test flags and targets without explicitly running the bazel command and some native support for remote caching. But I can see https://github.com/bazelbuild/vscode-bazel/blob/master/.bazelci/presubmit.yml is quite simple and requires more custom setup like installing nodejs, so I'm not against vscode-bazel moving to using GitHub Actions, it could be a better choice.

[cameron-martin]

I was hoping to start building this extension with bazel, so those features could be useful then. @meteorcloudy do you know if it's possible to get build artifacts out of bazelci, in this case for consuming within GitHub actions?

[meteorcloudy]

What are those build artifacts for? I hope it's not release artifacts?

[meteorcloudy]

BuildKite does support uploading artifacts after the build, which is later publicly accessible. See https://github.com/bazelbuild/continuous-integration/blob/35fc1da116a96631f83b45217ceb84e6ccee0106/buildkite/bazelci.py#L2659-L2665, but we need to implement a way to do that for artifacts at given paths in bazelci.py

[cameron-martin]

What are those build artifacts for? I hope it's not release artifacts?

The intention was to use them for this. Is that not a good idea?

[meteorcloudy]

https://buildkite.com/bazel should only be used for testing purposes, because by design it's an untrusted environment as we allow running PR presubmits without any review, so there are definitely security concerns when using it to build release artifacts. For release builds, we have a different Buildkite org: https://buildkite.com/bazel-trusted, but it's only used for google owned projects and only googlers can have access to it.

[cameron-martin]

Ah okay, is it okay to use GitHub actions to build release artifacts? I think rules_rust does this already if I'm not mistaken.

Also while you're here, do you know if there's any way to get automatically-created PRs (such as #360) past the CLA check? Maybe there's a way to use the @bazel-io account?

[meteorcloudy]

Ah okay, is it okay to use GitHub actions to build release artifacts? I think rules_rust does this already if I'm not mistaken.

Yes, I believe that's fine.

Also while you're here, do you know if there's any way to get automatically-created PRs (such as #360) past the CLA check?

I have no idea, but it's definitely possible. The @dependabot seems to pass the CLA in bazelbuild/continuous-integration#1938

Maybe there's a way to use the @bazel-io account?

Using @bazel-io is probably not possible since that requires generate a access token, we probably cannot share that with non-googler. I'm working on moving projects from bazelbuild to bazel-contrib. I think vscode-bazel should probably be one of them? And hopefully, that'll make such things easier in future.

[vogelsgesang]

I think vscode-bazel should probably be one of them?

I guess this depends on how much Google will be involved in the VS Code extension in the future...
Is Google internally using this extensions? Are there any plans to invest into it? If not, I guess moving this over to bazel-contrib and giving some interested non-Googlers the necessary permissions to sustain the VS Code extension by themselves, would make sense

[meteorcloudy]

Google isn't using this extension internally and we don't have plan or capacity to invest into it so far. I think moving this repo to bazel-contrib provides a better governance model for the repo and it won't prevent possible google contribution in future.

[vogelsgesang]

🤔 do you have any thoughts whether the Marketplace listing would also be handed over to the community? Or would that publishing still be done by Googlers?

[meteorcloudy]

@alexeagle is going through a similar process for setup-bazel on GH marketplace. I do hope the community maintainers can take over this process, so the publisher should change from the Bazel Team to Bazel Contrib or something else. But we can still help verify domain like (contrib.bazel.build) if necessary.

[jfirebaugh]

Related discussion about transferring repositories to bazel-contrib: https://github.com/orgs/bazelbuild/discussions/2.

[adam-azarchs]

This paragraph seems to maybe be out of date? I don't see the workflow using a "Build VS Code extension" action in use here, and frankly don't really see why you'd need one; the workflow is simple enough as is.

[vogelsgesang]

I confused the terms "action" and "workflow" here. It is the workflow which is named "Build VS Code extension". I udpated the README

[vogelsgesang]

Thanks for your review, @adam-azarchs. Very good points, I incorporated them into the updated version of this pull request

[vogelsgesang]

@jfirebaugh @cameron-martin How do we want to move ahead with this?

During the last couple of weeks, I think I got a better understanding of the pros/cons of Bazel CI vs. GitHub. Below my updated understanding.

Pro Bazel CI:

• Runs against commits to the main Bazel repository and not only for commits to the vscode-bazel repository. This allows rule authors to detect breakages earlier and allows Bazel contributors to be aware whether their changes are breaking any of the open-source rules.
• Remote cache: Bazel CI comes with a pre-configured remote cache. I am not aware of an equivalent offering which could be used part of GitHub actions.
• Machines provided by Google. Those machines seem to be more powerful than the default open-source Github workers (?)

Pro GitHub:

• Proper sandboxing: In contrast to Bazel CI, GitHub actions are properly sandboxed. In Bazel CI, a malicious PR could, e.g., poison the remote cache or do something similarly nefarious to inject malware into artifacts subsequently built on the same machine. Hence, we should not
• Changes to the CI config can be verified in a personal fork before merging. This is not possible with Bazel CI because it does not run for personal forks.
• Deeper integration with Github: GH_TOKEN available out-of-the-box. Can also trigger flows on "release" events and not only on "pre-submit"

Maybe I am still forgetting some pros & cons - not sure...

But under the assumption that this list is complete, I would still lean towards using GitHub actions going forward. I don't think this extension benefits much from the Bazel CI benefits, because

• it is rather small and quick to build. Hence, the remote caching and the faster build machines are not so important to this repo
• this extension is unlikely to be broken by upstream commits. We aren't writing rules, and are primarily using the bazel command line, which seems to be more stable than the Starlark interfaces

[cameron-martin]

I'd also lean towards using GitHub actions, mainly due to it being the only reasonable way to do automated releases and the downsides aren't significant.

[jfirebaugh]

I agree with your assessment @vogelsgesang and also support switching to GitHub Actions.

[vogelsgesang]

I updated this commit to fix the linter and also add the prettier-checks.
It should be ready for review again

[vogelsgesang]

See https://github.com/vogelsgesang/vscode-bazel/actions/runs/8742637975 for an up-to-date run

[vogelsgesang]

It seems to get this merged, we will first need to remove this project from Bazel CI and remove buildkite from the required checks. @meteorcloudy can you help with this?

[meteorcloudy]

I removed buildkite and added "Build VS Code extension" for presubmit check

[cameron-martin]

Why do we cut the upload URL at { here?

[vogelsgesang]

One of those instances where I am unhappy with my former self for not commenting the code properly... I just copied this over from https://github.com/vogelsgesang/hyper-ir-lsp/blob/main/.github/workflows/build.yml, but don't remember anymore why I wrote it this way in the first place.

I remember this to be some "poor man's template parsing". I think the upload_url was something like github.com/bazelbuild/vscode-bazel/releases/{some_placeholder} and I wanted to strip the placeholder.

I will reverse-engineer this later tonight and add comments to the code

[cameron-martin]

Using the gh command line tool is maybe clearer here, like this. You should be able to get the tag name as github.event.release.tag_name, and avoid any poor man's template parsing.

[alexeagle]

Did you consider an existing composite Action to do this? For bazel rulesets, we use this one: https://github.com/bazel-contrib/.github/blob/master/.github/workflows/release_ruleset.yaml#L69-L77

(In general I think it would be nice for this automation to be similar to that of rulesets where possible)

[vogelsgesang]

@alexeagle if understand correctly, the action-gh-release@v1 creates a new release. What we are trying to achieve here is slightly different, though: We want to attach artifacts to an already existing release

[vogelsgesang]

@cameron-martin good point. I updated the PR to use gh release upload instead

[cameron-martin]

Do we have a way of knowing if the release workflow works? I found this very hard on bazel-lsp - I only got feedback after doing a release, so it took quite a few iterations to get right.

[vogelsgesang]

I have been testing this flow on my personal fork and created a couple of "releases" there

https://github.com/vogelsgesang/vscode-bazel/releases/tag/test4

https://github.com/vogelsgesang/vscode-bazel/actions/runs/8760198382/job/24044751663