diff --git a/website/blog/2024-11-25_moon-v1.30.mdx b/website/blog/2024-11-25_moon-v1.30.mdx index 862913aefd..a8683524cb 100644 --- a/website/blog/2024-11-25_moon-v1.30.mdx +++ b/website/blog/2024-11-25_moon-v1.30.mdx @@ -79,6 +79,54 @@ python: ## Unstable self-hosted remote caching +This has been a request from the community for a very long time, and we get it, not every user wants +to store their build artifacts (not source code) in a third-party cloud provider. While we're proud +of our [moonbase service](/moonbase), it wasn't a viable option for many companies because of their +proprietary requirements. We spent a few months reworking moonbase to work as a self-hosted service, +so users can host it as on-prem solution, but it has been a very costly initiative. During this +process, we came to the conclusion that spending our time and resources on moonbase simply isn't +worth it, so we made the hard decision to sunset moonbase in the future. + +So what does that mean for remote caching? Simply put, you can now host your own remote caching +service! Instead of building a custom API for consumers to implement, we opted to implement the +[Bazel Remote Execution API](https://github.com/bazelbuild/remote-apis/blob/main/build/bazel/remote/execution/v2/remote_execution.proto), +which supports a content addressable storage (CAS) API, and is used by other popular build tools, +like Bazel, Buck, Pants, and more! + +Because we opted for a community solution, we can now focus all our efforts on [moon](/moon) and +[proto](/proto)! Additionally, adopting RE API allows you, the user, to use an off-the-shelf +solution, like [`bazel-remote`](https://github.com/buchgr/bazel-remote), instead of building your +own custom caching server! For example, to make use of remote caching, simply serve `bazel-remote`: + +```shell +bazel-remote --dir /path/to/moon-cache --max_size 10 --storage_mode uncompressed --grpc_address 0.0.0.0:9092 +``` + +And then configure the new [`unstable_remote`](/docs/config/workspace#unstable_remote) setting in +[`.moon/workspace.yml`](/docs/config/workspace). + +```yaml title=".moon/workspace.yml" +unstable_remote: + host: 'grpc://your-host.com:9092' +``` + +Pretty awesome right? Jump to the +[official remote caching](/docs/guides/remote-cache#self-hosted-unstable) documentation for more +information on this implementation. + +### Unsupported features + +Since this is a new feature, we're marking it as unstable, as it hasn't been thoroughly tested, and +_does not_ support the entire Bazel RE API. The following features _have not_ been implemented, but +will be in the future. + +- HTTP(S) host (we only support gRPC(S)) +- Digest hashing functions besides SHA256 +- Compression formats (we only support identity/uncompressed right now) +- Write/read bytestream for large blobs (4mb is the current limit) +- Better TLS/mTLS support (it has some issues) +- Directory blob types + ## New task graph and improved affected tracker In our [last release](./moon-v1.29#new-affected-projects-tracker), we announced a new affected diff --git a/website/docs/config/workspace.mdx b/website/docs/config/workspace.mdx index 3afe2d01f3..2a065a5a48 100644 --- a/website/docs/config/workspace.mdx +++ b/website/docs/config/workspace.mdx @@ -301,21 +301,6 @@ docker: Enable or disable experiments that alter core functionality. -### `strictProjectIds` - - - -When building the project graph, disallows referencing the original ID of a project (inferred from -the folder name) when the project has been renamed to a new ID using the [`id`](./project#id) -setting. - -Defaults to `false` but will be enabled to `true` in the future. - -```yaml title=".moon/workspace.yml" {1,2} -experiments: - strictProjectIds: true -``` - ## `extensions` @@ -497,9 +482,9 @@ notifier: webhookUrl: 'https://api.company.com/some/endpoint' ``` -## `remote` +## `unstable_remote` - + Configures a remote service, primarily for cloud-based caching of artifacts. Learn more about this in the [remote caching](../guides/remote-cache) guide. @@ -521,7 +506,7 @@ used to distinguish between the various instances on the host. This allows the s to serve and partition multiple moon repositories. Defaults to `moon-outputs`. ```yaml title=".moon/workspace.yml" {2} -remote: +unstable_remote: cache: instanceName: 'custom-dir-name' ``` @@ -536,7 +521,7 @@ The host URL to communicate with when uploading and download artifacts. Supports `grpc://` and `grpcs://` protocols. This field is required! ```yaml title=".moon/workspace.yml" {2} -remote: +unstable_remote: host: 'grpc://your-host.com:9092' ``` @@ -548,7 +533,7 @@ Connect to the host using server and client authentication with mTLS. This takes normal TLS. ```yaml title=".moon/workspace.yml" {3-7} -remote: +unstable_remote: # ... mtls: caCert: 'certs/ca.pem' @@ -598,7 +583,7 @@ The domain name in which to verify the TLS certificate. Connect to the host using server-only authentication with TLS. ```yaml title=".moon/workspace.yml" {3-5} -remote: +unstable_remote: # ... tls: cert: 'certs/ca.pem' diff --git a/website/docs/guides/remote-cache.mdx b/website/docs/guides/remote-cache.mdx index 093bc897d7..47d4b55eda 100644 --- a/website/docs/guides/remote-cache.mdx +++ b/website/docs/guides/remote-cache.mdx @@ -56,11 +56,12 @@ proxies, and more. ### Configure remote caching Once your service is running, you can enable remote caching by configuring the -[`remote`](../config/workspace#remote) settings in [`.moon/workspace.yml`](../config/workspace). At -minimum, the only setting that is required is `host`. +[`unstable_remote`](../config/workspace#unstable_remote) settings in +[`.moon/workspace.yml`](../config/workspace). At minimum, the only setting that is required is +`host`. ```yaml title=".moon/workspace.yml" -remote: +unstable_remote: host: 'grpc://your-host.com:9092' ``` @@ -73,14 +74,14 @@ tested. There's also [many](https://github.com/hyperium/tonic/issues/1652) ```yaml title=".moon/workspace.yml" # TLS -remote: +unstable_remote: host: 'grpcs://your-host.com:9092' tls: cert: 'certs/ca.pem' domain: 'your-host.com' # mTLS -remote: +unstable_remote: host: 'grpcs://your-host.com:9092' mtls: caCert: 'certs/ca.pem'