Holochain 0.4 upgrade guide

.cspell/custom-words.txt

@@ -36,6 +36,8 @@ rustflags
 rustup
 setgid
 setuid
+struct
+structs
 subl
 Tauri
 Ulhaq

.cspell/holochain-words.txt

@@ -11,6 +11,8 @@ Holochain
 Holochain's
 Holonix
 Lightningrod
+memproof
+memproofs
 Mythosthesia
 sharded
 Snapmail

.cspell/words-that-should-exist.txt

@@ -16,6 +16,7 @@ interoperating
 permissioned
 permissivity
 redistributable
+reserialize
 runtimes
 sandboxed
 sandboxing

netlify.toml

@@ -133,13 +133,13 @@
 # The first upgrade guide lived under getting started. It has been moved to its own area.
 [[redirects]]
     from = "/get-started/upgrade-holochain/"
-    to = "/resources/upgrade/upgrade-holochain-0.2/"
+    to = "/resources/upgrade/"
     status = 301
 
 # Every time there's a new breaking release that achieves 'recommended' status,
 # update this temporary redirect. /resources/upgrade/latest/ isn't in the site nav,
 # but it is a handy URL we can hand out without needing to look up what the proper URL is.
 [[redirects]]
     from = "/resources/upgrade/latest/"
-    to = "resources/upgrade/upgrade-holochain-0.2/"
+    to = "/resources/upgrade/upgrade-holochain-0.4/"
     status = 302

src/pages/get-started/install-advanced.md

@@ -19,18 +19,26 @@ If you want to learn more about how this setup works and how to create it manual
 
 [Flakes](https://wiki.nixos.org/wiki/Flakes) is an experimental but well-supported feature of the Nix package manager that makes it easier to manage dependencies consistently. [Enable flakes on your system.](https://wiki.nixos.org/wiki/Flakes#Enable_flakes_temporarily)
 
-The flake-based one-liner to get you an ad-hoc Holonix shell looks like this:
+### Entering an ad-hoc shell
+
+The flake-based one-liner to get you an ad-hoc Holonix shell (that is, not using a local flake file) looks like this:
 
 ```shell
 nix develop github:holochain/holonix
 ```
 
+#### Specifying a certain release
+
 The above one-liner will give you the latest version of Holochain from branch `main`. To get an ad-hoc shell with a specific version of Holochain, use the flag `--override-input versions <version_path>`.
 
 ```shell
 nix develop --override-input holochain "github:holochain/holochain?ref=main-0.4" github:holochain/holonix
 ```
 
+### Customizing the Holochain binary
+
+If you want to enable or disable certain Holochain features, such as unstable features, it's best to do this in a local flake file. [Read the 'Customized Holochain build'](https://github.com/holochain/holonix?tab=readme-ov-file#customized-holochain-build) on the Holonix readme to find out how. Keep in mind that, because you'll be creating a custom Holochain binary, you won't be able to take advantage of the package cache, so it'll take a while to compile Holochain on your machine.
+
 ### A gotcha with Flakes and Git
 
 The behavior of `nix` commands that rely on a `flake.nix` as its input such as `nix develop` can be counterintuitive in a git repository.

src/pages/resources/upgrade/index.md

@@ -8,6 +8,7 @@ hide:
 
 Upgrading between versions of Holochain can be a bit tricky! While Holochain is in beta, you can expect breaking changes between minor versions. To make upgrading your hApp easier, we create guides to help you move from one Holochain version to the next.
 
-* [Holochain Upgrade 0.1 → 0.2](/resources/upgrade/upgrade-holochain-0.2/)
-* [Holochain Upgrade 0.2 → 0.3](/resources/upgrade/upgrade-holochain-0.3/)
 * [Upgrading to the new Holonix](/resources/upgrade/upgrade-new-holonix/) (all Holochain versions)
+* [Holochain Upgrade 0.3 → 0.4](/resources/upgrade/upgrade-holochain-0.4/)
+* [Holochain Upgrade 0.2 → 0.3](/resources/upgrade/upgrade-holochain-0.3/)
+* [Holochain Upgrade 0.1 → 0.2](/resources/upgrade/upgrade-holochain-0.2/)

src/pages/resources/upgrade/upgrade-holochain-0.3.md

@@ -1,7 +1,5 @@
 ---
 title: Holochain Upgrade 0.2 → 0.3
-hide:
-  - toc
 ---
 
 ::: intro

src/pages/resources/upgrade/upgrade-holochain-0.4.md

@@ -0,0 +1,111 @@
+---
+title: Holochain Upgrade 0.3 → 0.4
+---
+
+::: intro
+For existing hApps that were written for Holochain 0.3, here's the guide to get you upgraded to 0.4.
+
+NOTE: [Holonix](/get-started/install-advanced/), our developer shell environment, has also been updated. We recommend you [upgrade to the new Holonix](/resources/upgrade-new-holonix/) at the same time!
+:::
+
+## Summary
+
+Here are all the breaking changes you need to know about in order to update your app for Holochain 0.4:
+
+* Some unstable features are now behind feature flags.
+* The `OpenChain` and `CloseChain` actions have been modified.
+* The database encryption scheme has changed, and the directory structure has moved around.
+* The `InstallApp` admin API endpoint has had its request payload changed.
+* `CloneCellId`, used as an input argument to clone manipulation functions, has changed.
+* The `CountersigningSuccess` signal supplies the action hash, not the entry hash.
+* The `AppInfo` admin endpoint's response contains new statuses related to deferred memproofs.
+* The WebRTC signalling server architecture has changed, along with new server addresses and self-hosted server binaries.
+* Deprecated `OpType` and `op::to_type` have been removed from the HDI.
+* Enums are serialized differently.
+
+## Unstable features removed by default
+
+The biggest change for 0.4 is that some unstable features aren't compiled into official Holochain releases by default. We're doing this to focus on a stable core Holochain with the minimal feature set that most projects need. The removed features are still available as Rust feature flags, so you can compile a custom Holochain binary if you need them. Here's the full list, along with their feature flags:
+
+* **[Countersigning](/concepts/10_countersigning/) `unstable-countersigning`**: Allows agents to temporarily lock their chains in order to agree to write the same entry to their respective chains.
+* **DPKI / DeepKey `unstable-dpki`**: Implements a distributed public key infrastructure, allowing agents to manage their keys and confirm continuity of their identity across devices and source chains. Read the [DeepKey readme](https://github.com/holochain/deepkey) for more info.
+* **[DHT sharding](/concepts/4_dht/) `unstable-sharding`**: Lessens the load of each peer in large network, allowing them to take responsibility for validating and storing only a subset of a DHT's entire data set.
+* **[Warrants](/concepts/7_validation#remembering-validation-results) `unstable-warrants`**: Spreads news of validation failure around the network, allowing peers to recognize bad actors and take action against them, even if they haven't directly interacted with them.
+* **Chain head coordination `chc`**: Developed for Holo, allows source chain changes to be copied from one device to another without causing chain forks. (This pre-existing feature flag is no longer enabled by default.)
+* **HDI/HDK functions `unstable-functions`**:
+    * Related to countersigning (enable `unstable-countersigning` if you want to use these):
+        * [`accept_countersigning_preflight_request`](https://github.com/holochain/holochain/blob/holochain-0.4.0-rc.2/crates/hdk/src/countersigning.rs#L3-L22)
+    * Related to DPKI (enable `unstable-dpki` if you want to use these, which are both new to 0.4):
+        * [`get_agent_key_lineage`](https://github.com/holochain/holochain/blob/holochain-0.4.0-rc.2/crates/hdk/src/agent.rs#L10-L20)
+    * Related to block lists (allowing peers in a hApp to selectively block others who haven't necessarily produced invalid data or forked their source chain):
+        * `block_agent`
+        * `unblock_agent`
+    * Other:
+        * [`schedule`](https://github.com/holochain/holochain/blob/holochain-0.4.0-rc.2/crates/hdk/src/time.rs#L67-L137)
+        * An unimplemented `sleep` function has been removed completely
+
+Read the [Holonix readme](https://github.com/holochain/holonix?tab=readme-ov-file#customized-holochain-build) to find out how to compile a custom Holochain build with these flags enabled.
+
+`unstable-functions` is a flag used by both the Holochain conductor _and_ the [`hdi`](https://docs.rs/hdi/latest/hdi) and [`hdk`](https://docs.rs/hdk/latest/hdk) crates, and some of the functions also need other Holochain features enabled (e.g., `is_same_agent` requires a conductor with `unstable-dpki` enabled; see the list above). If you want to use them, you'll need to edit your zome crates' `Cargo.toml` files and make sure that users are running your custom conductor binary with the right features enabled. If you compile your zomes without `unstable-functions` enabled, users with the flag(s) enabled in Holochain will still be able to use your hApp, but if you enable it, users with the flag(s) disabled won't be able to use your hApp. If you use any of the unstable functions, note that the conductor will also need to have the corresponding feature enabled.
+
+## [DHT sharding](/concepts/4_dht/) is disabled by default
+
+This feature needs more performance and correctness testing before it's production-ready. With the `unstable-sharding` feature flag disabled by default, your conductor config's `gossip_arc_clamping` must now be set to either `"full"` or `"empty"`, and the previous default `"none"` will cause a conductor startup error. `"gossip_dynamic_arcs"` is also ignored.
+
+It's unknown exactly what might happen if nodes with DHT sharding disabled try to gossip in the same network as nodes without DHT sharding. Presumably this is still possible, but it might cause unexpected behaviors!
+
+## `OpenChain` and `CloseChain` actions changed
+
+If you're one of the rare folks who have been using these two actions, the structs have changed. `CloseChain::new_dna_hash` has been replaced with [`new_target`](https://docs.rs/holochain_zome_types/0.4.0-rc.1/holochain_zome_types/action/struct.CloseChain.html), which has a type of `Option<MigrationTarget>`. [This new type](https://docs.rs/holochain_zome_types/0.4.0-rc.1/holochain_zome_types/prelude/enum.MigrationTarget.html) is an enum of `Dna(DnaHash)` or `Agent(AgentHash)`.
+
+[`OpenChain::prev_dna_hash`](https://docs.rs/holochain_zome_types/0.4.0-rc.1/holochain_zome_types/action/struct.OpenChain.html) has been changed to match; its type is a non-optional `MigrationTarget`. It also gets a new field, `close_hash`, which is the `ActionHash` of the corresponding `CloseChain` action.
+
+## Dynamic database encryption, folder structure changes
+
+Previous conductor and keystore SQLite databases used a hardcoded encryption key; v0.4 now uses a dynamic key. This means you won't be able to import data from v0.3.
+
+The database names have also changed; any file prefix that matches the folder it's in (e.g., `<root>/authorized/authorized-<dna_hash>-<agent_key>.sqlite`) is dropped, along with the `.sqlite` file extension.
+
+## `InstallApp` agent key is optional
+
+The `agent_key` field in the [`InstallApp` payload](https://docs.rs/holochain_types/0.4.0-rc.2/holochain_types/app/struct.InstallAppPayload.html) is now optional, and a key will be generated if you don't supply one. If your app is used to generating agent keys and supplying them on install, you shouldn't need to change your code; just update the client library and it'll work.
+
+## `CloneCellId` changes
+
+The `CloneCellId::CellId` enum variant has become [`DnaHash`](https://docs.rs/holochain_zome_types/0.4.0-rc.1/holochain_zome_types/clone/enum.CloneCellId.html) and contains, naturally, a `DnaHash` value. This type is used when enabling, disabling, or deleting clones from the app API or a coordinator zome.
+
+## `CountersigningSuccess` information changed
+
+The `CountersigningSuccess` signal sent to the client now gives the action hash of the successful countersigned commit rather than its entry hash.
+
+## New data in [`AppInfo` response](https://docs.rs/holochain_conductor_api/0.4.0-rc.2/holochain_conductor_api/struct.AppInfo.html)
+
+Holochain 0.4 introduces a new flow for app installation that lets an agent supply a membrane proof later with a new [`ProvideMemproofs` endpoint](https://docs.rs/holochain_conductor_api/0.4.0-rc.2/holochain_conductor_api/enum.AppRequest.html#variant.ProvideMemproofs). This allows them to get their freshly generated agent key, submit it to some sort of membrane proof service, and get a membrane proof.
+
+Because of this, after the membrane proof has been provided, the [`DisabledAppReason`](https://docs.rs/holochain_types/0.4.0-rc.2/holochain_types/app/enum.DisabledAppReason.html) enum, used in the [`AppInfo`](https://docs.rs/holochain_conductor_api/0.4.0-rc.2/holochain_conductor_api/struct.AppInfo.html) response, will be a new `NotStartedAfterProvidingMemproofs` variant until the app is started.
+
+## WebRTC signalling server change
+
+The old signalling server has been replaced with a new one called [`sbd`](https://github.com/holochain/sbd/). If you're hosting your own server, switch to a [new `sbd` server binary](https://github.com/holochain/sbd/tags). If you're using Holo's public infrastructure, switch from `wss://signal.holo.host` to `wss://sbd-0.main.infra.holo.host` in your conductor config file. (The new URL will automatically be in any conductor config generated by the dev tools for 0.4.)
+
+## Deprecated validation op functionality removed
+
+In your integrity zome's validation functions, you deal with DHT operations, or ops. They are somewhat complex, so [`FlatOp`](https://docs.rs/hdi/latest/hdi/flat_op/enum.FlatOp.html) was introduced to make things simpler. It was originally called `OpType`, and until now that old name was a deprecated alias of `FlatOp`. The old type has finally been removed, along with the `Op::to_type` method (use [`OpHelper::flattened`](https://docs.rs/hdi/latest/hdi/op/trait.OpHelper.html#tymethod.flattened) instead).
+
+## Change in enum serialization
+
+The default serialization for bare enum variants has changed. Previously, they would look like this (JSON representation):
+
+```json
+{
+    "variant1": null
+}
+```
+
+Now they look like this:
+
+```json
+"variant1"
+```
+
+(Enum variants with data still follow the `{ "variant": <data> }` pattern.)