Skip to content

Commit

Permalink
Finish blog post.
Browse files Browse the repository at this point in the history
  • Loading branch information
milesj committed Nov 25, 2024
1 parent 10fc51f commit f30185f
Show file tree
Hide file tree
Showing 3 changed files with 60 additions and 26 deletions.
48 changes: 48 additions & 0 deletions website/blog/2024-11-25_moon-v1.30.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,54 @@ python:

## Unstable self-hosted remote caching

This has been a request from the community for a very long time, and we get it, not every user wants
to store their build artifacts (not source code) in a third-party cloud provider. While we're proud
of our [moonbase service](/moonbase), it wasn't a viable option for many companies because of their
proprietary requirements. We spent a few months reworking moonbase to work as a self-hosted service,
so users can host it as on-prem solution, but it has been a very costly initiative. During this
process, we came to the conclusion that spending our time and resources on moonbase simply isn't
worth it, so we made the hard decision to sunset moonbase in the future.

So what does that mean for remote caching? Simply put, you can now host your own remote caching
service! Instead of building a custom API for consumers to implement, we opted to implement the
[Bazel Remote Execution API](https://github.com/bazelbuild/remote-apis/blob/main/build/bazel/remote/execution/v2/remote_execution.proto),
which supports a content addressable storage (CAS) API, and is used by other popular build tools,
like Bazel, Buck, Pants, and more!

Because we opted for a community solution, we can now focus all our efforts on [moon](/moon) and
[proto](/proto)! Additionally, adopting RE API allows you, the user, to use an off-the-shelf
solution, like [`bazel-remote`](https://github.com/buchgr/bazel-remote), instead of building your
own custom caching server! For example, to make use of remote caching, simply serve `bazel-remote`:

```shell
bazel-remote --dir /path/to/moon-cache --max_size 10 --storage_mode uncompressed --grpc_address 0.0.0.0:9092
```

And then configure the new [`unstable_remote`](/docs/config/workspace#unstable_remote) setting in
[`.moon/workspace.yml`](/docs/config/workspace).

```yaml title=".moon/workspace.yml"
unstable_remote:
host: 'grpc://your-host.com:9092'
```

Pretty awesome right? Jump to the
[official remote caching](/docs/guides/remote-cache#self-hosted-unstable) documentation for more
information on this implementation.

### Unsupported features

Since this is a new feature, we're marking it as unstable, as it hasn't been thoroughly tested, and
_does not_ support the entire Bazel RE API. The following features _have not_ been implemented, but
will be in the future.

- HTTP(S) host (we only support gRPC(S))
- Digest hashing functions besides SHA256
- Compression formats (we only support identity/uncompressed right now)
- Write/read bytestream for large blobs (4mb is the current limit)
- Better TLS/mTLS support (it has some issues)
- Directory blob types

## New task graph and improved affected tracker

In our [last release](./moon-v1.29#new-affected-projects-tracker), we announced a new affected
Expand Down
27 changes: 6 additions & 21 deletions website/docs/config/workspace.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -301,21 +301,6 @@ docker:

Enable or disable experiments that alter core functionality.

### `strictProjectIds`<VersionLabel version="1.29.4" />

<HeadingApiLink to="/api/types/interface/ExperimentsConfig#strictProjectIds" />

When building the project graph, disallows referencing the original ID of a project (inferred from
the folder name) when the project has been renamed to a new ID using the [`id`](./project#id)
setting.

Defaults to `false` but will be enabled to `true` in the future.

```yaml title=".moon/workspace.yml" {1,2}
experiments:
strictProjectIds: true
```

## `extensions`<VersionLabel version="1.20.0" />

<HeadingApiLink to="/api/types/interface/WorkspaceConfig#extensions" />
Expand Down Expand Up @@ -497,9 +482,9 @@ notifier:
webhookUrl: 'https://api.company.com/some/endpoint'
```

## `remote`<VersionLabel version="1.30.0" />
## `unstable_remote`<VersionLabel version="1.30.0" />

<HeadingApiLink to="/api/types/interface/WorkspaceConfig#remote" />
<HeadingApiLink to="/api/types/interface/WorkspaceConfig#unstable_remote" />

Configures a remote service, primarily for cloud-based caching of artifacts. Learn more about this
in the [remote caching](../guides/remote-cache) guide.
Expand All @@ -521,7 +506,7 @@ used to distinguish between the various instances on the host. This allows the s
to serve and partition multiple moon repositories. Defaults to `moon-outputs`.

```yaml title=".moon/workspace.yml" {2}
remote:
unstable_remote:
cache:
instanceName: 'custom-dir-name'
```
Expand All @@ -536,7 +521,7 @@ The host URL to communicate with when uploading and download artifacts. Supports
`grpc://` and `grpcs://` protocols. This field is required!

```yaml title=".moon/workspace.yml" {2}
remote:
unstable_remote:
host: 'grpc://your-host.com:9092'
```

Expand All @@ -548,7 +533,7 @@ Connect to the host using server and client authentication with mTLS. This takes
normal TLS.

```yaml title=".moon/workspace.yml" {3-7}
remote:
unstable_remote:
# ...
mtls:
caCert: 'certs/ca.pem'
Expand Down Expand Up @@ -598,7 +583,7 @@ The domain name in which to verify the TLS certificate.
Connect to the host using server-only authentication with TLS.

```yaml title=".moon/workspace.yml" {3-5}
remote:
unstable_remote:
# ...
tls:
cert: 'certs/ca.pem'
Expand Down
11 changes: 6 additions & 5 deletions website/docs/guides/remote-cache.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,12 @@ proxies, and more.
### Configure remote caching

Once your service is running, you can enable remote caching by configuring the
[`remote`](../config/workspace#remote) settings in [`.moon/workspace.yml`](../config/workspace). At
minimum, the only setting that is required is `host`.
[`unstable_remote`](../config/workspace#unstable_remote) settings in
[`.moon/workspace.yml`](../config/workspace). At minimum, the only setting that is required is
`host`.

```yaml title=".moon/workspace.yml"
remote:
unstable_remote:
host: 'grpc://your-host.com:9092'
```
Expand All @@ -73,14 +74,14 @@ tested. There's also [many](https://github.com/hyperium/tonic/issues/1652)
```yaml title=".moon/workspace.yml"
# TLS
remote:
unstable_remote:
host: 'grpcs://your-host.com:9092'
tls:
cert: 'certs/ca.pem'
domain: 'your-host.com'

# mTLS
remote:
unstable_remote:
host: 'grpcs://your-host.com:9092'
mtls:
caCert: 'certs/ca.pem'
Expand Down

0 comments on commit f30185f

Please sign in to comment.