diff --git a/.cspell/custom-words.txt b/.cspell/custom-words.txt index c74c3fee3..287781ede 100644 --- a/.cspell/custom-words.txt +++ b/.cspell/custom-words.txt @@ -27,9 +27,12 @@ libatk libcups libgtk libnss +Multihash +multihash NixOS nixpkgs pkgs +pubkey QUIC rustc rustflags diff --git a/.cspell/words-that-should-exist.txt b/.cspell/words-that-should-exist.txt index 97c97ed7f..d0a79021b 100644 --- a/.cspell/words-that-should-exist.txt +++ b/.cspell/words-that-should-exist.txt @@ -12,7 +12,10 @@ filesystem filesystems howto howtos +ified interoperating +lifecycle +lifecycles permissioned permissivity redistributable diff --git a/.gitignore b/.gitignore index 3843cc651..00da44800 100644 --- a/.gitignore +++ b/.gitignore @@ -111,4 +111,8 @@ dist # My 11ty specifics _site/ client-side-compiled/ -styles-compiled/ \ No newline at end of file +styles-compiled/ + +# A place to put a local scaffolded DNA so you can test code for the build guide +# without having to have two projects/folderss open in your editor. +code_test/ diff --git a/11ty-extensions/eleventy-transforms.js b/11ty-extensions/eleventy-transforms.js index 6229ba48f..9913c963e 100644 --- a/11ty-extensions/eleventy-transforms.js +++ b/11ty-extensions/eleventy-transforms.js @@ -47,7 +47,9 @@ export default function(eleventyConfig) { pre.className += ' hljs-container'; const code = pre.querySelector('code'); const maybeLanguage = code.className.match(/(?<=\blanguage-)[A-Za-z0-9_-]+/); - const blockText = he.decode(code.textContent); + let blockText = he.decode(code.textContent); + // Erase cspell directives from sample code. + blockText = blockText.replace(/(#|\/\/|\/\*)\s+(cspell|spell-?checker):\s*[a-z-]+(\s+\*\/)?/gmi, ""); if (maybeLanguage) { code.innerHTML = hljs.highlight(blockText, {language: maybeLanguage[0]}).value; } else { diff --git a/src/pages/_data/navigation/mainNav.json5 b/src/pages/_data/navigation/mainNav.json5 index e23e94cb2..395bdfc37 100644 --- a/src/pages/_data/navigation/mainNav.json5 +++ b/src/pages/_data/navigation/mainNav.json5 @@ -26,7 +26,9 @@ }, { title: "Build", url: "/build/", children: [ { title: "Working with Data", url: "/build/working-with-data/", children: [ + { title: "Identifiers", url: "/build/identifiers/" }, { title: "Entries", url: "/build/entries/" }, + { title: "Links, Paths, and Anchors", url: "/build/links-paths-and-anchors/" }, ]}, ] }, diff --git a/src/pages/build/entries.md b/src/pages/build/entries.md index aac91a8f7..d5f5a651e 100644 --- a/src/pages/build/entries.md +++ b/src/pages/build/entries.md @@ -15,22 +15,29 @@ An entry is always paired with an **entry creation action** that tells you who a The pairing of an entry and the action that created it is called a **record**, which is the basic unit of data in a Holochain application. +## Scaffold an entry type and CRUD API + +The Holochain dev tool command `hc scaffold entry-type ` generates the code for a simple entry type and a CRUD API. It presents an interface that lets you define a struct and its fields, then asks you to choose whether to implement update and delete functions for it along with the default create and read functions. + ## Define an entry type -Each entry has a **type**, which your application code uses to make sense of the entry's bytes. Our [HDI library](https://docs.rs/hdi/latest/hdi/) gives you macros to automatically define, serialize, and deserialize entry types to and from any Rust struct or enum that [`serde`](https://docs.rs/serde/latest/serde/) can handle. +Each entry has a **type**. This lets your application make sense of what would otherwise be a blob of arbitrary bytes. Our [HDI library](https://docs.rs/hdi/latest/hdi/) gives you macros to automatically define, serialize, and deserialize typed entries to and from any Rust struct or enum that [`serde`](https://docs.rs/serde/latest/serde/) can handle. Entry types are defined in an [**integrity zome**](/resources/glossary/#integrity-zome). To define an [`EntryType`](https://docs.rs/hdi/latest/hdi/prelude/enum.EntryType.html), use the [`hdi::prelude::hdk_entry_helper`](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_entry_helper.html) macro on your Rust type: ```rust use hdi::prelude::*; +#[hdk_entry_helper] +pub struct Director(pub String); + #[hdk_entry_helper] pub struct Movie { - title: String, - director: String, - imdb_id: Option, - release_date: Timestamp, - box_office_revenue: u128, + pub title: String, + pub director_hash: EntryHash, + pub imdb_id: Option, + pub release_date: Timestamp, + pub box_office_revenue: u128, } ``` @@ -42,13 +49,18 @@ In order to dispatch validation to the proper integrity zome, Holochain needs to use hdi::prelude::*; #[hdk_entry_defs] +// This macro is required by hdk_entry_defs. +#[unit_enum(UnitEntryTypes)] enum EntryTypes { - Movie(Movie), - // other types... + Director(Director), + Movie(Movie), + // other types... } ``` -### Configuring an entry type +This also gives you an enum that you can use later when you're storing app data. Under the hood, an entry type consists of two bytes --- an integrity zome index and an entry def index. These are required whenever you want to write an entry. Instead of having to remember those values every time you store something, your coordinator zome can just import and use this enum, which already knows how to convert each entry type to the right IDs. + +### Configure an entry type Each variant in the enum should hold the Rust type that corresponds to it, and is implicitly marked with an `entry_def` proc macro which, if you specify it explicitly, lets you configure the given entry type further: @@ -59,47 +71,51 @@ Each variant in the enum should hold the Rust type that corresponds to it, and i use hdi::prelude::*; #[hdk_entry_defs] +#[unit_enum(UnitEntryTypes)] enum EntryTypes { - #[entry_def(required_validations = 7, )] + Director(Director), + + #[entry_type(required_validations = 7, )] Movie(Movie), // You can reuse your Rust type in another entry type if you like. In this // example, `HomeMovie` also (de)serializes to/from the `Movie` struct, but // is actually a different entry type with different visibility, and can be // subjected to different validation rules. - #[entry_def(visibility = "private", )] + #[entry_type(visibility = "private", )] HomeMovie(Movie), } ``` -This also gives you an enum that you can use later when you're storing app data. This is important because, under the hood, an entry type consists of two bytes -- an integrity zome index and an entry def index. These are required whenever you want to write an entry. Instead of having to remember those values every time you store something, your coordinator zome can just import and use this enum, which already knows how to convert each entry type to the right IDs. - ## Create an entry Most of the time you'll want to define your create, read, update, and delete (CRUD) functions in a [**coordinator zome**](/resources/glossary/#coordinator-zome) rather than the integrity zome that defines it. This is because a coordinator zome is easier to update in the wild than an integrity zome. -Create an entry by calling [`hdk::prelude::create_entry`](https://docs.rs/hdk/latest/hdk/prelude/fn.create_entry.html). If you used `hdk_entry_helper` and `hdk_entry_defs` macro in your integrity zome (see [Define an entry type](#define-an-entry-type)), you can use the entry types enum you defined, and the entry will be serialized and have the correct integrity zome and entry type indexes added to it. +Create an entry by calling [`hdk::prelude::create_entry`](https://docs.rs/hdk/latest/hdk/entry/fn.create_entry.html). If you used `hdk_entry_helper` and `hdk_entry_defs` macro in your integrity zome (see [Define an entry type](#define-an-entry-type)), you can use the entry types enum you defined, and the entry will be serialized and have the correct integrity zome and entry type indexes added to it. ```rust use hdk::prelude::*; -use chrono::Date; // Import the entry types and the enum defined in the integrity zome. use movie_integrity::*; +use chrono::DateTime; let movie = Movie { - title: "The Good, the Bad, and the Ugly", - director: "Sergio Leone" - imdb_id: Some("tt0060196"), - release_date: Timestamp::from(Date::Utc("1966-12-23")), - box_office_revenue: 389_000_000, + title: "The Good, the Bad, and the Ugly".to_string(), + director_hash: EntryHash::from_raw_36(vec![ /* hash of 'Sergio Leone' entry */ ]), + imdb_id: Some("tt0060196".to_string()), + release_date: Timestamp::from( + DateTime::parse_from_rfc3339("1966-12-23")? + .to_utc() + ), + box_office_revenue: 389_000_000, }; -let create_action_hash: ActionHash = create_entry( +let create_action_hash = create_entry( // The value you pass to `create_entry` needs a lot of traits to tell // Holochain which entry type from which integrity zome you're trying to // create. The `hdk_entry_defs` macro will have set this up for you, so all // you need to do is wrap your movie in the corresponding enum variant. - &EntryTypes::Movie(movie.clone()), + &EntryTypes::Movie(movie), )?; ``` @@ -135,20 +151,24 @@ Update an entry creation action by calling [`hdk::prelude::update_entry`](https: ```rust use hdk::prelude::*; -use chrono::Date; use movie_integrity::*; +use chrono::DateTime; let movie2 = Movie { - title: "The Good, the Bad, and the Ugly", - director: "Sergio Leone" - imdb_id: Some("tt0060196"), - release_date: Timestamp::from(Date::Utc("1966-12-23")), - box_office_revenue: 400_000_000, + title: "The Good, the Bad, and the Ugly".to_string(), + director_hash: EntryHash::from_raw_36(vec![ /* hash of 'Sergio Leone' entry */ ]), + imdb_id: Some("tt0060196".to_string()), + release_date: Timestamp::from( + DateTime::parse_from_rfc3339("1966-12-23")? + .to_utc() + ), + // Corrected from 389_000_000 + box_office_revenue: 400_000_000, }; -let update_action_hash: ActionHash = update_entry( +let update_action_hash = update_entry( create_action_hash, - &EntryTypes::Movie(movie2.clone()), + &EntryTypes::Movie(movie2), )?; ``` @@ -217,12 +237,12 @@ Delete an entry creation action by calling [`hdk::prelude::delete_entry`](https: ```rust use hdk::prelude::*; -let delete_action_hash: ActionHash = delete_entry( +let delete_action_hash = delete_entry( create_action_hash, )?; ``` -As with an update, this does _not_ actually remove data from the source chain or the DHT. Instead, a [`Delete` action](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/struct.Delete.html) is authored, which attaches to the entry creation action and marks it as 'dead'. An entry itself is only considered dead when all entry creation actions that created it are marked dead, and it can become live again in the future if a _new_ entry creation action writes it. Dead data can still be retrieved with [`hdk::prelude::get_details`](https://docs.rs/hdk/latest/hdk/prelude/fn.get_details.html) (see below). +As with an update, this does _not_ actually remove data from the source chain or the DHT. Instead, a [`Delete` action](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/struct.Delete.html) is authored, which attaches to the entry creation action and marks it as deleted. An entry itself is only considered deleted when _all_ entry creation actions that created it are marked deleted, and it can become live again in the future if a _new_ entry creation action writes it. Deleted data can still be retrieved with [`hdk::prelude::get_details`](https://docs.rs/hdk/latest/hdk/prelude/fn.get_details.html) (see below). In the future we plan to include a 'purge' functionality. This will give agents permission to actually erase an entry from their DHT store, but not its associated entry creation action. @@ -260,22 +280,23 @@ You can use any of these identifiers as a field in your entry types to model a m ## Retrieve an entry -### By record only +### As a single record -Get a record by calling [`hdk::prelude::get`](https://docs.rs/hdk/latest/hdk/prelude/fn.get.html) with the hash of its entry creation action. The return value is a Result<[holochain_integrity_types::record::Record](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/record/struct.Record.html)>. +Get a record by calling [`hdk::prelude::get`](https://docs.rs/hdk/latest/hdk/prelude/fn.get.html) with the hash of either its entry creation action. The return value is a Result<[holochain_integrity_types::record::Record](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/record/struct.Record.html)>. -You can also pass an entry hash to `get`, and the record returned will contain the _oldest live_ entry creation action that wrote it. +You can also pass an _entry hash_ to `get`, and the record returned will contain the _oldest live_ entry creation action that wrote it. ```rust use hdk::prelude::*; use movie_integrity::*; -let maybe_record: Option = get( +let maybe_record = get( action_hash, - // Get the data and metadata directly from the DHT. You can also specify - // `GetOptions::content()`, which only accesses the DHT if the data at the - // supplied hash doesn't already exist locally. - GetOptions::latest() + // Get the data and metadata directly from the DHT, falling back to local + // storage if it can't access peers. + // You can also specify `GetOptions::local()`, which only accesses the local + // storage. + GetOptions::network() )?; match maybe_record { @@ -288,7 +309,7 @@ match maybe_record { // `Record` type, but in this simple example we'll skip that and assume // that the action hash does reference an action with entry data // attached to it. - let maybe_movie: Option = record.entry().to_app_option(); + let maybe_movie = record.entry().to_app_option()?; match maybe_movie { Some(movie) => debug!( @@ -302,7 +323,7 @@ match maybe_record { } } None => debug!("Movie record not found"), -} +}; ``` ### All data, actions, and links at an address @@ -315,14 +336,14 @@ To get a record and all the updates, deletes, and outbound links associated with use hdk::prelude::*; use movie_integrity::*; -let maybe_details: Option
= get_details( +let maybe_details = get_details( action_hash, - GetOptions::latest() + GetOptions::network() )?; match maybe_details { Some(Details::Record(record_details)) => { - let maybe_movie: Option = record.entry().to_app_option(); + let maybe_movie: Option = record_details.record.entry().to_app_option()?; match maybe_movie { Some(movie) => debug!( "Movie record {}, created on {}, was updated by {} agents and deleted by {} agents", @@ -335,7 +356,7 @@ match maybe_details { } } _ => debug!("Movie record not found"), -} +}; ``` #### Entries @@ -348,24 +369,22 @@ use movie_integrity::*; let maybe_details: Option
= get_details( entry_hash, - GetOptions::latest() + GetOptions::network() )?; match maybe_details { Some(Details::Entry(entry_details)) => { - let maybe_movie: Option = entry_details.entry + let maybe_movie = entry_details.entry .as_app_entry() - .clone() - .try_into() - .ok(); + .map(|b| Movie::try_from(b.into_sb())) + .transpose()?; match maybe_movie { Some(movie) => debug!( - "Movie {} was written by {} agents, updated by {} agents, and deleted by {} agents. Its DHT status is currently {}.", + "Movie {} was written by {} agents, updated by {} agents, and deleted by {} agents.", movie.title, entry_details.actions.len(), entry_details.updates.len(), - entry_details.deletes.len(), - entry_details.entry_dht_status + entry_details.deletes.len() ), None => debug!("Movie entry couldn't be retrieved"), } @@ -374,13 +393,9 @@ match maybe_details { } ``` -## Scaffolding an entry type and CRUD API - -The Holochain dev tool command `hc scaffold entry-type ` generates the code for a simple entry type and a CRUD API. It presents an interface that lets you define a struct and its fields, then asks you to choose whether to implement update and delete functions for it along with the default create and read functions. - ## Community CRUD libraries -If the scaffolder doesn't support your desired functionality, or is too low-level, there are some community-maintained libraries that offer opinionated and high-level ways to work with entries. Some of them also offer permissions management. +There are some community-maintained libraries that offer opinionated and high-level ways to work with entries. Some of them also offer permissions management. * [rust-hc-crud-caps](https://github.com/spartan-holochain-counsel/rust-hc-crud-caps) * [hdk_crud](https://github.com/lightningrodlabs/hdk_crud) @@ -388,14 +403,16 @@ If the scaffolder doesn't support your desired functionality, or is too low-leve ## Reference -* [hdi::prelude::hdk_entry_helper](https://docs.rs/hdi/latest/hdi/attr.hdk_entry_helper.html) -* [hdi::prelude::hdk_entry_defs](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_entry_defs.html) -* [hdi::prelude::entry_def](https://docs.rs/hdi/latest/hdi/prelude/entry_def/index.html) -* [hdk::prelude::create_entry](https://docs.rs/hdk/latest/hdk/entry/fn.create_entry.html) -* [hdk::prelude::update_entry](https://docs.rs/hdk/latest/hdk/entry/fn.update_entry.html) -* [hdi::prelude::delete_entry](https://docs.rs/hdk/latest/hdk/entry/fn.delete_entry.html) +* [`hdi::prelude::hdk_entry_helper`](https://docs.rs/hdi/latest/hdi/attr.hdk_entry_helper.html) +* [`hdi::prelude::hdk_entry_defs`](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_entry_defs.html) +* [`hdi::prelude::entry_def`](https://docs.rs/hdi/latest/hdi/prelude/entry_def/index.html) +* [`hdk::prelude::create_entry`](https://docs.rs/hdk/latest/hdk/entry/fn.create_entry.html) +* [`hdk::prelude::update_entry`](https://docs.rs/hdk/latest/hdk/entry/fn.update_entry.html) +* [`hdi::prelude::delete_entry`](https://docs.rs/hdk/latest/hdk/entry/fn.delete_entry.html) ## Further reading * [Core Concepts: CRUD actions](/concepts/6_crud_actions/) +* [Build Guide: Identifiers](/build/identifiers/) +* [Build Guide: Links, Paths, and Anchors](/build/links-paths-and-anchors/) * [CRDT.tech](https://crdt.tech), a resource for learning about conflict-free replicated data types diff --git a/src/pages/build/identifiers.md b/src/pages/build/identifiers.md new file mode 100644 index 000000000..ca14ecb33 --- /dev/null +++ b/src/pages/build/identifiers.md @@ -0,0 +1,265 @@ +--- +title: "Identifiers" +--- + +::: intro +Data in Holochain is **addressable content**, which means that it's retrieved using an address that's derived from the data itself --- usually the **hash** of the content. +::: + +## Address types + +The address of most data is the [Blake2b-256](https://www.blake2.net/) hash of its bytes. This goes for [**actions** and most **entries**](/build/working-with-data/#entries-actions-and-records-primary-data), along with a few other types which we'll talk about in a moment. + +All addresses are 39 bytes long and are [multihash-friendly](https://www.multiformats.io/multihash/). Generally, you don't need to know how to construct an address. You usually get it via functions that store data and types that hold data. But when you do see a hash in the wild, this is what it's made out of: + +| Multihash prefix | Hash | DHT location | +|------------------|----------|--------------| +| 3 bytes | 32 bytes | 4 bytes | + +The four-byte DHT location is calculated from the 32 bytes of the hash and is used in routing to the right peer. The three-byte multihash prefix will be one of the following: + +| Hash type | [`holo_hash`](https://docs.rs/holo_hash/latest/holo_hash/#types) type | Prefix in Base64 | +|-----------|-------------------------------------------------------------------------------------|------------------| +| DNA | [`DnaHash`](https://docs.rs/holo_hash/latest/holo_hash/type.DnaHash.html) | `hC0k` | +| agent ID | [`AgentPubKey`](https://docs.rs/holo_hash/latest/holo_hash/type.AgentPubKey.html) | `hCAk` | +| action | [`ActionHash`](https://docs.rs/holo_hash/latest/holo_hash/type.ActionHash.html) | `hCkk` | +| entry | [`EntryHash`](https://docs.rs/holo_hash/latest/holo_hash/type.EntryHash.html) | `hCEk` | +| external | [`ExternalHash`](https://docs.rs/holo_hash/latest/holo_hash/type.ExternalHash.html) | `hC8k` | + +There are also a couple of composite types, [`AnyDhtHash`](https://docs.rs/holo_hash/latest/holo_hash/type.AnyDhtHash.html) and [`AnyLinkableHash`](https://docs.rs/holo_hash/latest/holo_hash/type.AnyLinkableHash.html). + +Here's an overview of the five types above, plus two composite types: + +* `DnaHash` is the hash of the DNA bundle (including any DNA modifiers passed in at installation or cloning time), and is the [unique identifier for the network](/build/working-with-data/#storage-locations-and-privacy). +* `AgentPubKey` is the public key of a participant in a network. +* `ActionHash` is the hash of a structure called an [action](/build/working-with-data/#entries-actions-and-records-primary-data) that records a participant's act of storing or changing private or shared data. +* `EntryHash` is the hash of an arbitrary blob of bytes called an [entry](/build/entries/), which contains application or system data. (Note: there's a special system entry called [`Agent`](https://docs.rs/holochain_zome_types/latest/holochain_zome_types/enum.Entry.html#variant.Agent), which holds the agent's public key; the hash function returns the public key itself, _not_ its hash.) +* `ExternalHash` is the ID of a resource that exists outside the database, such as the hash of an IPFS resource or the public key of an Ethereum wallet. Holochain doesn't care about its value, as long as it's 32 bytes long. There's no content stored at the address; it simply serves as an anchor to attach [links](/build/links-paths-and-anchors/) to. +* Composite types --- if one of the types above is eligible, it can be converted into one of these two types via the `.into()` method. Functions that take the below types will implicitly convert from the above types. + * `AnyDhtHash` is the hash of any kind of addressable content (actions, entries, and agent public keys). Any + * `AnyLinkableHash` is the hash of anything that can be linked to or from (that is, all of the above). + +## Getting hashes + +Because Holochain's graph DHT is all about connecting hashes to other hashes, here's how you get hashes. + +!!! info Hashing functions aren't built in +To keep compiled zomes small, there are no hashing functions built into the HDI or HDK. Most of the time, you'll get the hash you need from a property in a data structure or the return value of a host function. In the rare cases when you need to compute a hash, use the [host functions in the `hdi::hash` module](https://docs.rs/hdi/latest/hdi/hash/index.html#functions). There are functions for hashing actions, entries, and arbitrary vectors of bytes. +!!! + +### Action + +Any CRUD host function that records an action on an agent's source chain, such as `create`, `update`, `delete`, `create_link`, and `delete_link`, returns the hash of the action. You can use this in the fields of other entries or in links, in either the same function call or another function call. + +If you have a variable that contains a [`hdk::prelude::Record`](https://docs.rs/hdk/latest/hdk/prelude/struct.Record.html), you can get its hash using the [`action_address`](https://docs.rs/hdk/latest/hdk/prelude/struct.Record.html#method.action_address) method or the [`as_hash` method on its `signed_action` property](https://docs.rs/hdk/latest/hdk/prelude/struct.SignedHashed.html#method.as_hash): + +```rust +let action_hash_from_record = record.action_address().to_owned(); +let action_hash_from_signed_action = record.signed_action.as_hash().to_owned(); +assert_eq!(action_hash_from_record, action_hash_from_signed_action); +``` + +If you have a variable that contains a [`hdk::prelude::Action`](https://docs.rs/hdk/latest/hdk/prelude/enum.Action.html), you need to calculate its hash using the [`hdi::hash::hash_action`](https://docs.rs/hdi/latest/hdi/hash/fn.hash_action.html) host function: + +```rust +use hdi::hash::*; + +let action_hash_from_action = hash_action(action)?; +assert_eq!(action_hash_from_signed_action, action_hash_from_action); +``` + +(But it's worth pointing out that if you have an action in a variable, it's probably because you just retrieved it by hash, which means you already know the hash.) + +To get the hash of an entry creation action from an action that deletes or updates it, match on the [`Action::Update`](https://docs.rs/hdk/latest/hdk/prelude/enum.Action.html#variant.Update) or [`Action::Delete`](https://docs.rs/hdk/latest/hdk/prelude/enum.Action.html#variant.Delete) action variants and access the appropriate field: + +```rust +use holochain_integrity_types::action::*; + +if let Action::Update(action_data) = action { + let replaced_action_hash = action_data.original_action_address; + // Do some things with the original action. +} else if let Action::Delete(action_data) = action { + let deleted_action_hash = action_data.deletes_address; + // Do some things with the deleted action. +} +``` + +### Entry + +To get the hash of an entry, first construct an instance of the entry type that you [defined in the integrity zome](/build/entries/#define-an-entry-type), then pass it through the [`hdk::hash::hash_entry`](https://docs.rs/hdk/latest/hdk/hash/fn.hash_entry.html) function. (You don't actually have to write the entry to a source chain to get the entry hash.) + +```rust +use hdk::hash::*; +use movie_integrity::*; +use chrono::DateTime; + +let movie = Movie { + title: "The Good, the Bad, and the Ugly", + director_entry_hash: EntryHash::from_raw_36(vec![/* hash of Sergio Leone entry */]), + imdb_id: Some("tt0060196"), + release_date: Timestamp::from( + DateTime::parse_from_rfc3339("1966-12-23")? + .to_utc() + ), + box_office_revenue: 389_000_000, +}; +let movie_entry_hash = hash_entry(movie)?; +``` + +To get the hash of an entry from the action that created it, call the action's [`entry_hash`](https://docs.rs/hdk/latest/hdk/prelude/enum.Action.html#method.entry_hash) method. It returns an optional value, because not all actions have associated entries. + +```rust +let maybe_entry_hash = action.entry_hash(); +``` + +If you know that your action is an entry creation action, you can get the entry hash from its `entry_hash` field: + +```rust +let entry_creation_action: EntryCreationAction = action.into()?; +let entry_hash = entry_creation_action.entry_hash; +``` + +To get the hash of an entry from a record, you can get it from the contained action: + +```rust +let entry_hash_from_action = record.action().entry_hash()?; +``` + +Finally, to get the hash of an entry from an action that updates or deletes it, match the action to the appropriate variant and access the corresponding field: + +```rust +if let Action::Update(action_data) = action { + let replaced_entry_hash = action_data.original_entry_address; +} else if let Action::Delete(action_data) = action { + let deleted_entry_hash = action_data.deletes_entry_address; +} +``` + +### Agent + +An agent's ID is just their public key, and an entry for their ID is stored on the DHT. The hashing function for an `Agent` system entry just returns the literal value of the public key. This is an awkward way of saying that you reference an agent using their public key! + +An agent can get their own ID by calling [`hdk::prelude::agent_info`](https://docs.rs/hdk/latest/hdk/info/fn.agent_info.html). + +```rust +use hdk::prelude::*; + +let my_id = agent_info()?.agent_initial_pubkey; +``` + +All actions have their author's ID as a field. You can get this field by calling the action's [`author`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html#method.author) method: + +```rust +let author_id = action.author(); +``` + +### External reference + +An external reference is just any 32-byte identifier. Holochain doesn't care if it's an IPFS hash, an Ethereum wallet, a very short URL, or the name of your pet cat. But because it comes from outside of a DHT, it's up to your application to decide how to handle it. Typically, an external client such as a UI would supply external references from a source it has access to, such as an HTTP API or a form field. + +To construct an external hash from 32 raw bytes, first you need to enable the `hashing` feature in the `holo_hash` crate. In your zome's `Cargo.toml` add this line: + +```diff +... +[dependencies] +hdk = { workspace = true } +serde = { workspace = true } ++ # Replace the following version number with whatever your project is ++ # currently using -- search your root `Cargo.lock` for "holo_hash" to find it. ++ holo_hash = { version = "=0.4.0", features = ["hashing"] } +... +``` + +Then you can construct an `ExternalHash`: + +```rust +use holo_hash::*; +let ipfs_movie_poster_hash = ExternalHash::from_raw_32(vec![/* bytes of external hash */]); +``` + +### DNA + +There is one global hash that everyone knows, and that's the hash of the DNA itself. You can get it by calling [`hdk::prelude::dna_info`](https://docs.rs/hdk/latest/hdk/info/fn.dna_info.html). + +```rust +use hdk::prelude::*; + +let dna_hash = dna_info()?.hash; +``` + +## Using addresses + +### In DHT data + +To reference an address in your entry data, define a field in your entry that can hold the right kind of address. The HDK will take care of serialization and deserialization for you. The following entry type has two fields that take different kinds of address. + +```rust +use hdi::prelude::*; + +#[hdk_entry_helper] +pub struct MovieLoan { + movie_hash: EntryHash, + lent_to: AgentPubKey, + loan_duration_seconds: i64, +} + +// Remember to create a variant in your `EntryTypes` enum for this new type! +``` + +To reference an address in your links, pass it directly to the `create_link` function: + +```rust +use hdk::prelude::*; + +let movie_to_loan_action_hash = create_link( + movie_hash, + movie_loan_hash, + LinkTypes::MovieToLoan, + () +)?; +``` + +Read more about [entries](/build/entries/) and [links](/build/links-paths-and-anchors/). + +## The unpredictability of action hashes + +There are a few important things to know about action hashes: + +* You can't know an action's hash until you've written the action, because the action contains the current system time at the moment of writing. +* When you write an action, you can specify "relaxed chain top ordering". We won't go into the details here, but when you use it, the action hash may change after the function completes. +* A function that writes actions is _atomic_, which means that all writes fail or succeed together. + +Because of these three things, it's unsafe to depend on the value or even existence of an action hash within the same function that writes it. Here are some 'safe usage' notes: + +* You may safely use the hash of an action you've just written as data in another action in the same function (e.g., in a link or an entry that contains the hash in a field), as long as you're not using relaxed chain top ordering. +* The same is also true of action hashes in your function's return value. +* Don't communicate the action hash with the front end, another cell, or another peer on the network via a remote function call or [signal](/concepts/9_signals/) _from within the same function that writes it_, in case the write fails. Instead, do your communicating in a follow-up step. The easiest way to do this is by implementing [a callback called `post_commit`](https://docs.rs/hdk/latest/hdk/#internal-callbacks) which receives a vector of all the actions that the function wrote. + + + +## Reference + +* [`holo_hash` types](https://docs.rs/holo_hash/latest/holo_hash/#types) +* [`holo_hash::DnaHash`](https://docs.rs/holo_hash/latest/holo_hash/type.DnaHash.html) +* [`holo_hash::AgentPubKey`](https://docs.rs/holo_hash/latest/holo_hash/type.AgentPubKey.html) +* [`holo_hash::ActionHash`](https://docs.rs/holo_hash/latest/holo_hash/type.ActionHash.html) +* [`holo_hash::EntryHash`](https://docs.rs/holo_hash/latest/holo_hash/type.EntryHash.html) +* [`holo_hash::ExternalHash`](https://docs.rs/holo_hash/latest/holo_hash/type.ExternalHash.html) +* [`holo_hash::AnyDhtHash`](https://docs.rs/holo_hash/latest/holo_hash/type.AnyDhtHash.html) +* [`holo_hash::AnyLinkableHash`](https://docs.rs/holo_hash/latest/holo_hash/type.AnyLinkableHash.html) +* [`holochain_integrity_types::record::Record`](https://docs.rs/hdk/latest/hdk/prelude/struct.Record.html) +* [`holochain_integrity_types::record::SignedHashed`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/record/struct.SignedHashed.html) (e.g., an action in a record) +* [`holochain_integrity_types::action::Action`](https://docs.rs/hdk/latest/hdk/prelude/enum.Action.html) +* [`holochain_integrity_types::action::Update`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/struct.Update.html) data struct +* [`holochain_integrity_types::action::Delete`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/struct.Delete.html) data struct +* [`hdk::prelude::agent_info`](https://docs.rs/hdk/latest/hdk/info/fn.agent_info.html) +* [`holochain_integrity_types::action::Action#author`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html#method.author) +* [`holo_hash::HoloHash

#from_raw_32`](https://docs.rs/holo_hash/latest/src/holo_hash/hash.rs.html#217-219) (must be enabled by `hashing` feature flag) +* [`hdi::info::dna_info`](https://docs.rs/hdi/latest/hdi/info/fn.dna_info.html) + +## Further reading + +* [Explanation of Holochain hash format](https://docs.rs/hdi/latest/hdi/hash/index.html) +* [Build Guide: Working With Data](/build/working-with-data/) +* [Build Guide: Entries](/build/entries/) +* [Build Guide: Links, Paths, and Anchors](/build/links-paths-and-anchors/) \ No newline at end of file diff --git a/src/pages/build/index.md b/src/pages/build/index.md index ae43f1695..d69836e2f 100644 --- a/src/pages/build/index.md +++ b/src/pages/build/index.md @@ -16,5 +16,7 @@ This Build Guide organizes everything you need to know about developing Holochai ### Topics {data-no-toc} * [Overview](/build/working-with-data/) --- general concepts related to working with data in Holochain -* [Entries](/build/entries/) --- creating, reading, updating, and deleting +* [Identifiers](/build/identifiers) --- working with hashes and other unique IDs +* [Entries](/build/entries/) --- defining, creating, reading, updating, and deleting data +* [Links, Paths, and Anchors](/build/links-paths-and-anchors/) --- creating relationships between data ::: \ No newline at end of file diff --git a/src/pages/build/links-paths-and-anchors.md b/src/pages/build/links-paths-and-anchors.md new file mode 100644 index 000000000..66354a7d9 --- /dev/null +++ b/src/pages/build/links-paths-and-anchors.md @@ -0,0 +1,281 @@ +--- +title: "Links, Paths, and Anchors" +--- + +::: intro +A **link** connects two addresses in an application's shared database, forming a graph database on top of the underlying hash table. Links can be used to connect pieces of [addressable content](/resources/glossary/#addressable-content) in the database or references to addressable content outside the database. + +An **anchor** is a pattern of linking from a well-known base address. Our SDK includes a library for creating hierarchies of anchors called **paths**, allowing you to create things like collections, pagination, indexes, and taxonomies. +::: + +## Turning a hash table into a graph database + +A Holochain application's database is, at heart, just a big key-value store --- or more specifically, a hash table. You store and retrieve content by hash. This is useful if you already know the hash of the content you want to retrieve, but it isn't helpful if you don't. + +A piece of content itself can contain a hash as part of its data structure, and that's great for modelling a _many-to-one relationship_. And if the number of things the content points to is small and doesn't change often, you can model a _many-to-many relationship_ using a field that contains an array of hashes. But at a certain point this becomes hard to manage, especially if that list regularly changes or gets really large. + +To help with this, Holochain also lets you attach **links** as metadata to an address in the database. You can then retrieve a full or filtered list of links from that address in order to discover more addressable content. In this way you can build up a traversable graph database. + +### Define a link type + +Every link has a type that you define in an integrity zome, just like [an entry](/build/entries/#define-an-entry-type). Links are simple enough that they're committed as an action with no associated entry. Here's what a link creation action contains, in addition to the [common action fields](/build/working-with-data/#entries-actions-and-records-primary-data): + +* A **base**, which is the address the link is attached to and _points from_ +* A **target**, which is the address the link _points to_ +* A **type** +* An optional **tag** that can hold a small amount of arbitrary bytes, up to 1 kb + +You can use the tag as link 'content' to further qualify the link, provide a summary of data about the target to save on DHT queries, or build a starts-with search index. But unlike an entry's content, the HDK doesn't provide a macro that automatically deserializes the link tag's content into a Rust type. + +[Just as with entries](/build/entries/#define-an-entry-type), Holochain needs to know about your link types in order to dispatch validation to the right integrity zome. You can do this by implementing a `link_types` callback function, and the easiest way to do this is to add the [`hdi::prelude::hdk_link_types`](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_link_types.html) macro to an enum that defines all your link types: + +```rust +use hdi::prelude::*; + +// Generate a `link_types` function that returns a list of definitions. +#[hdk_link_types] +enum LinkTypes { + DirectorToMovie, + GenreToMovie, + IpfsMoviePoster, + MovieReview, + MovieToLoan, + // Note: the following link types will become useful when we talk about + // paths and anchors later. + MovieByFirstLetterAnchor, + MovieByFirstLetter, + // other types... +} +``` + +## Create a link + +As with entries, you'll normally want to store your link CRUD code in a [**coordinator zome**](/resources/glossary/#coordinator-zome), not an integrity zome. You can read about why in the page on [entries](/build/entries/#create-an-entry). + +Create a link by calling [`hdk::prelude::create_link`](https://docs.rs/hdk/latest/hdk/link/fn.create_link.html). If you used the `hdk_link_types` macro in your integrity zome (see [Define a link type](#define-a-link-type)), you can use the link types enum you defined: + +```rust +use hdk::prelude::*; +// Import the link types enum defined in the integrity zome. +use movie_integrity::*; + +let director_entry_hash = EntryHash::from_raw_36(vec![ /* bytes of the hash of the Sergio Leone entry */ ]); +let movie_entry_hash = EntryHash::from_raw_36(vec![ /* bytes of the hash of The Good, The Bad, and The Ugly entry */ ]); + +let create_link_action_hash = create_link( + director_entry_hash, + movie_entry_hash, + LinkTypes::DirectorToMovie, + // Cache a bit of the target entry in this link's tag, as a search index. + vec!["year:1966".as_bytes()].into() +)?; +``` + +Links can't be updated; they can only be created or deleted. Multiple links with the same base, target, type, and tag can be created, and they'll be considered separate links for retrieval and deletion purposes. + +## Delete a link + +Delete a link by calling [`hdk::prelude::delete_link`](https://docs.rs/hdk/latest/hdk/link/fn.delete_link.html) with the create-link action's hash. + +```rust +use hdk::prelude::*; + +let delete_link_action_hash = delete_link( + create_link_action_hash +); +``` + +A link is considered ["dead"](/build/working-with-data/#deleted-dead-data) (deleted but retrievable if asked for explicitly) once its creation action has at least one delete-link action associated with it. As with entries, dead links can still be retrieved with [`hdk::prelude::get_details`](https://docs.rs/hdk/latest/hdk/prelude/fn.get_details.html) or [`hdk::prelude::get_link_details`](https://docs.rs/hdk/latest/hdk/link/fn.get_link_details.html) (see next section). + +## Retrieve links + +Get all the _live_ (undeleted) links attached to a hash with the [`hdk::prelude::get_links`](https://docs.rs/hdk/latest/hdk/link/fn.get_links.html) function. The input is complicated, so use [`hdk::link::builder::GetLinksInputBuilder`](https://docs.rs/hdk/latest/hdk/link/builder/struct.GetLinksInputBuilder.html) to build it. + +```rust +use hdk::prelude::*; +use movie_integrity::*; + +let director_entry_hash = EntryHash::from_raw_36(vec![/* hash of Sergio Leone's entry */]); +let movies_by_director = get_links( + GetLinksInputBuilder::try_new(director_entry_hash, LinkTypes::DirectorToMovie)? + .get_options(GetStrategy::Network) + .build() +)?; +let movie_entry_hashes = movies_by_director + .iter() + .filter_map(|link| link.target.into_entry_hash()) + .collect::>(); +``` + +If you want to filter the returned links by tag, pass some bytes to the input builder's `tag_prefix` method. You'll get back a vector of links whose tag starts with those bytes. + +```rust +let movies_in_1960s_by_director = get_links( + GetLinksInputBuilder::try_new(director_entry_hash, LinkTypes::DirectorToMovie)? + .get_options(GetStrategy::Network) + .tag_prefix("year:196".as_bytes().to_owned().into()) + .build() +)?; +``` + +To get all live _and deleted_ links, along with any deletion actions, use [`hdk::prelude::get_link_details`](https://docs.rs/hdk/latest/hdk/link/fn.get_link_details.html). + +```rust +use hdk::prelude::*; +use movie_integrity::*; + +let movies_plus_deleted = get_link_details( + director_entry_hash, + LinkTypes::DirectorToMovie, + None, + GetOptions::network() +)?; +``` + +### Count links + +If all you need is a _count_ of matching links, use [`hdk::prelude::count_links`](https://docs.rs/hdk/latest/hdk/prelude/fn.count_links.html). It has a different input with more options for querying (we'll likely update the inputs of `get_links` and `count_links` to match in the future). + +```rust +use hdk::prelude::*; +use movie_integrity::*; + +let my_id = agent_info()?.agent_initial_pubkey; +let today = sys_time()?; +let number_of_reviews_written_by_me_in_last_month = count_links( + LinkQuery::new( + movie_entry_hash, + LinkTypes::MovieReview.try_into_filter()? + ) + .after(Timestamp(today.as_micros() - 1000 * 1000 * 60 * 60 * 24 * 30)) + .before(today) + .author(my_id) +)?; +``` + +!!! info Links are counted locally +Currently `count_links` retrieves all link hashes from the remote peer, then counts them locally. So it is less network traffic than a `get_links` request, but more network traffic than just sending an integer. +!!! + +## Anchors and paths + +Sometimes the best way to discover a link base is to build it into the application's code. You can create an **anchor**, a well-known address (like the hash of a string constant) to attach links to. This can be used to simulate collections or tables in your graph database. + +While you can build this yourself, this is such a common pattern that the HDK implements it for you in the [`hdk::hash_path`](https://docs.rs/hdk/latest/hdk/hash_path/index.html) module. With it, you can create **paths**, which are hierarchies of anchors. + +!!! info Avoiding DHT hot spots +Too many links on a single base address creates extra work for the peers responsible for it. Use a hierarchy of paths to split the links into appropriate 'buckets' and spread the work around. We'll give an example of that [below](#paths-in-depth). +!!! + +### Scaffold a simple collection anchor + +The scaffolding tool can create a 'collection', which is a one-level path that serves as an anchor for entries of a given type, along with all the functionality that creates and deletes links from that anchor to its entries: + +```bash +hc scaffold collection +``` + +Follow the prompts to choose the entry type, name the link types and anchor, and define the scope of the collection, which can be either: + +* all entries of type, or +* entries of type by author + +### Paths in depth + +When you want to create more complex collections, you'll want to use the paths library directly. + +Create a path by constructing a [`hdk::hash_path::path::Path`](https://docs.rs/hdk/latest/hdk/hash_path/path/struct.Path.html) struct, hashing it, and using the hash as a link base. The string of the path is a simple [domain-specific language](https://docs.rs/hdk/latest/hdk/hash_path/path/struct.Path.html#impl-From%3C%26str%3E-for-Path), in which dots denote sections of the path. + +```rust +use hdk::hash_path::path::*; +use movie_integrity::*; + +// This will create a two-level path that looks like: +// "movies_by_first_letter" → "g" +let path_to_movies_starting_with_g = Path::from("movies_by_first_letter.g") + // A path requires a link type that you've defined in the integrity zome. + // Here, we're using the `MovieByFirstLetterAnchor` type that we created. + .typed(LinkTypes::MovieByFirstLetterAnchor)?; + +// Make sure it exists before attaching links to it -- if it already exists, +// ensure() will have no effect. +path_to_movies_starting_with_g.ensure()?; + +let create_link_hash = create_link( + path_to_movies_starting_with_g.path_entry_hash()?, + movie_entry_hash, + LinkTypes::MovieByFirstLetter, + () +)?; +``` + +Retrieve all the links on a path by constructing the path, then calling `get_links` with its hash and link type: + +```rust +use hdk::hash_path::path::*; +use movie_integrity::*; + +let path_to_movies_starting_with_g = Path::from("movies_by_first_letter.g"); +let links_to_movies_starting_with_g = get_links( + // A path doesn't need to have a type in order to compute its hash. + GetLinksInputBuilder::try_new( + path_to_movies_starting_with_g.path_entry_hash()?, + LinkTypes::MovieByFirstLetter + )? +)?; +``` + +Retrieve all child paths of a path by constructing the parent path and calling its `children_paths()` method: + +```rust +use hdk::hash_path::path::*; +use movie_integrity::*; + +let parent_path = Path::from("movies_by_first_letter") + .typed(LinkTypes::MovieByFirstLetterAnchor)?; +let all_first_letter_paths = parent_path.children_paths()?; +// Do something with the children. Note: this would be expensive to do in +// practice, because each child needs a separate DHT query. +let links_to_all_movies = all_first_letter_paths + .iter() + .map(|path| get_links( + GetLinksInputBuilder::try_new( + path.path_entry_hash()?, + LinkTypes::MovieByFirstLetter + )? + .build() + )) + // Fail on the first failure. + .collect::, _>>()? + .into_iter() + .flatten() + .collect(); +``` + +## Community path libraries + +* [holochain-prefix-index](https://github.com/holochain-open-dev/holochain-prefix-index) --- An index for starts-with text searching, useful for type-ahead username or hashtag lookup. +* [holochain-time-index](https://github.com/holochain-open-dev/holochain-time-index) --- A 'bucketed' time-based index, where you can choose the resolution of the time slices. + +## Reference + +* [`hdi::prelude::hdk_link_types`](https://docs.rs/hdi/latest/hdi/prelude/attr.hdk_link_types.html) +* [`hdk::prelude::create_link`](https://docs.rs/hdk/latest/hdk/link/fn.create_link.html) +* [`hdk::prelude::delete_link`](https://docs.rs/hdk/latest/hdk/link/fn.delete_link.html) +* Getting hashes from data + * [`hdk::prelude::Record#action_address`](https://docs.rs/hdk/latest/hdk/prelude/struct.Record.html#method.action_address) + * [`hdk::prelude::HasHash`](https://docs.rs/hdk/latest/hdk/prelude/trait.HasHash.html) + * [`hdk::prelude::Action`](https://docs.rs/hdk/latest/hdk/prelude/enum.Action.html) (contains fields with hashes of referenced data in them) + * [`hdk::hash::hash_entry`](https://docs.rs/hdk/latest/hdk/hash/fn.hash_entry.html) + * [`hdk::prelude::agent_info`](https://docs.rs/hdk/latest/hdk/info/fn.agent_info.html) + * [`hdk::prelude::dna_info`](https://docs.rs/hdk/latest/hdk/info/fn.dna_info.html) +* [`hdk::prelude::get_links`](https://docs.rs/hdk/latest/hdk/link/fn.get_links.html) +* [`hdk::prelude::get_link_details`](https://docs.rs/hdk/latest/hdk/link/fn.get_link_details.html) +* [`hdk::prelude::count_links`](https://docs.rs/hdk/latest/hdk/prelude/fn.count_links.html) +* [`hdk::hash_path`](https://docs.rs/hdk/latest/hdk/hash_path/index.html) +* [`hdk::prelude::anchor`](https://docs.rs/hdk/latest/hdk/prelude/fn.anchor.html) + +## Further reading + +* [Core Concepts: Links and Anchors](https://developer.holochain.org/concepts/5_links_anchors/) +* [Build Guide: Identifiers](/build/identifiers) \ No newline at end of file diff --git a/src/pages/build/working-with-data.md b/src/pages/build/working-with-data.md index 58442b28a..985097ec8 100644 --- a/src/pages/build/working-with-data.md +++ b/src/pages/build/working-with-data.md @@ -2,75 +2,104 @@ title: Working With Data --- +::: topic-list +### In this section {data-no-toc} + +* Working with Data (this page) + * [Identifiers](/build/identifiers/) --- understanding and using addresses of different types + * [Entries](/build/entries/) --- creating, reading, updating, and deleting + * [Links, Paths, and Anchors](/build/links-paths-and-anchors/) --- creating and deleting +::: + ::: intro -Holochain is, at its most basic, a framework for building **graph databases** on top of **content-addressed storage** that are validated and stored by **networks of peers**. Each peer contributes to the state of this database by publishing **actions** to an event journal stored on their device called their **source chain**. The source chain can also be used to hold private state. +Holochain is, at its most basic, a framework for building **graph databases** on top of **content-addressed storage** that is validated and stored by **networks of peers**. Each peer contributes to the state of this database by publishing **actions** to an event journal called their **source chain**, which is stored on their device. The source chain can also be used to hold private data. ::: ## Entries, actions, and records: primary data -Data in Holochain takes the shape of a [**record**](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/record/struct.Record.html). Different kinds of records have different purposes, but the thing common to all records is the [**action**](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html): one participant's intent to manipulate their own state or the application's shared database state in some way. All actions contain: +Data in Holochain takes the shape of a [**record**](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/record/struct.Record.html). Different kinds of records have different purposes, but the thing common to all records is the [**action**](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/action/enum.Action.html): one participant's attempt to manipulate their own state and/or the application's shared database state in some way. All actions contain: * The **agent ID** of the author * A timestamp * The type of action -* The hash of the previous action in the author's history of state changes, called their [**source chain**](/concepts/3_source_chain/) (note: the first action in their chain doesn't contain this field, as it's the first) +* The hash of the previous action in the author's history of state changes, called their [**source chain**](/concepts/3_source_chain/) (note: the first action in their chain doesn't contain this field) * The index of the action in the author's source chain, called the **action seq** -Some actions also contain a **weight**, which is a calculation of the cost of storing the action and can be used for rate limiting. (Note: weighting and rate limiting isn't implemented yet.) +Some actions also contain a **weight**, which is a calculation of the cost of storing the action and can be used for spam prevention. (Note: weighting isn't implemented yet.) The other important part of a record is the **entry**. Not all action types have an entry to go along with them, but those that do, `Create` and `Update`, are called **entry creation actions** and are the main source of data in an application. -It's generally most useful to _think about a **record** (entry plus creation action) as the primary unit of data_. This is because the action holds useful context about when an entry was written and by whom. -One entry written by two actions is considered to be the same piece of content, but when paired with their respective actions into records, each record is guaranteed to be unique. +It's generally most useful to _think about a **record** (entry plus creation action) as the primary unit of data_. This is because the action holds useful context about when an entry was written and by whom. A unique entry, no matter how many times it's written, is considered to be one piece of content: - -
authors
authors
authors
creates
creates
creates
Alice
Action 1
Bob
Action 2
Carol
Action 3
Entry
 +```json +"hello" // entry ID is hCEkhy0q54imKYjEpFdLTncbqAaCEGK3LCE+7HIA0kGIvTw // cspell:disable-line +``` + +But that entry, paired with its respective creation actions into records, can be treated as two pieces of content: -Entries and actions are both **addressable content**, which means that they're retrieved by their address --- which is usually the hash of their data. All addresses are 32-byte identifiers. +```json +{ + "create": { + "author": "hCAkKUej3Mcu+40AjNGcaID2sQA6uAUcc9hmJV9XIdwUJUE", // cspell:disable-line + "timestamp": 1732314805000000, + "entry_hash": "hCEkhy0q54imKYjEpFdLTncbqAaCEGK3LCE+7HIA0kGIvTw" // "hello" entry // cspell:disable-line + } +} // action ID is hCkkDHBZjU1a7L3gm6/qhImbWG6KG4Oc2ZiWDyfPSGoziBs // cspell:disable-line +``` -There are four types of addressable content: +```json +{ + "create": { + "author": "hCAk4R0sY+orZRxeFwqFTQSrhalgY+W2pLEJ5mihgY3CE7A", // cspell:disable-line + "timestamp": 1481614210000000, + "entry_hash": "hCEkhy0q54imKYjEpFdLTncbqAaCEGK3LCE+7HIA0kGIvTw" // "hello" entry // cspell:disable-line + } +} // action ID is hCkk1Oqnmn/xDVFNS+L2Z2PuQf9nN1/FmoAewlA8SV10jb8 // cspell:disable-line +``` -* An **entry** is an arbitrary blob of bytes, and its address is the hash of that blob. It has an **entry type**, which your application uses to deserialize it, validate it, and give it meaning. -* An **agent ID** is the public key of a participant in an application. Its address is the same as its content. -* An **action** stores action data for a record, and its address is the hash of the serialized action content. -* An **external reference** is the ID of a resource that exists outside the database, such as the hash of an IPFS resource or the public key of an Ethereum address. There's no content stored at the address; it simply serves as an anchor to attach [links](#links) to. +(Note: these samples are simplified and JSON-ified to focus on the bits that matter.) -### Storage locations + +
authors
authors
authors
creates
creates
creates
Alice
Action 1
Bob
Action 2
Carol
Action 3
Entry
 -Addressable content can either be: +## The graph DHT: Holochain's shared database -* **Private**, stored on the author's device in their [source chain](#individual-state-histories-as-public-records), or -* **Public**, stored in the application's shared graph database and accessible to all participants. +Each application creates a shared [**graph database**](https://en.wikipedia.org/wiki/Graph_database), where content is connected together by [**links**](/concepts/5_links_anchors/). The underlying data store for this database is a [distributed hash table](/resources/glossary/#distributed-hash-table-dht) or DHT, which is just a big key/value store. Primary content (entries and actions) is stored and retrieved by its identifier or address (usually its cryptographic hash), so we can call it **addressable content**. Then, the graph is built by attaching **links** and other kinds of **metadata** to those same addresses. -All actions are public, while entries can be either public or [private](/build/entries/#configuring-an-entry-type). External references hold neither public nor private content, but merely point to content outside the database. +The application's users all share responsibility for storing and validating this database and modifications to it. -## Shared graph database +### Storage locations and privacy -Shared data in a Holochain application is represented as a [**graph database**](https://en.wikipedia.org/wiki/Graph_database) of nodes connected by edges called [**links**](/concepts/5_links_anchors/). Any kind of addressable content can be used as a graph node. +Each [DNA](/concepts/2_application_architecture/#dna) creates a network of peers who participate in storing pieces of that DNA's database, which means that each DNA's database (and the [source chains](#individual-state-histories-as-public-records) that contribute to it) is completely separate from all others. This creates a per-network privacy for shared data. On top of that, entries can either be: -This database is stored in a [distributed hash table (DHT)](/resources/glossary/#distributed-hash-table-dht), which is a key/value store whose data is distributed throughout the network. In a Holochain application, the users themselves become network participants and help keep the DHT alive by storing a portion of the hash table. +* [**Private**](/build/entries/#configure-an-entry-type), stored encrypted on the author's device in their source chain in an encrypted database and accessible to them only, or +* **Public**, stored in the graph database and accessible to all participants. + +All actions are public. ### Links -A link is a piece of metadata attached to an address, the **base**, and points to another address, the **target**. It has a **link type** that gives it meaning in the application just like an entry type, as well as an optional **tag** that can store arbitrary application data. +A [link](/build/links-paths-and-anchors/) is a piece of metadata attached to an address, the **base**, and points to another address, the **target**. It has a **link type** that gives it meaning in the application just like an entry type, as well as an optional **tag** that can store arbitrary application data.

type: artist_album

type: artist_album_by_release_date tag: 1966-01-17

type: artist_album

type: artist_album_by_release_date tag: 1970-01-26

Simon & Garfunkel
Sounds of Silence
Bridge over Troubled Water
 -When a link's base and target don't exist as addressable content in the database, they're considered external references whose data isn't accessible to your application's back end. +When a link's base and target don't exist as addressable content in the database, they're considered **external references**, and it's up to your front end to decide how to handle them.

type: eth_wallet_to_ipfs_profile_photo

hC8kafe9...7c12
hC8kd01f...84ce
 ### CRUD metadata graph -Holochain has a built-in [**create, read, update, and delete (CRUD)** model](/concepts/6_crud_actions/). Data in the graph database and participants' local state cannot be modified or deleted, so these kinds of mutation are simulated by attaching metadata to existing data that marks changes to its status. This builds up a graph of the history of a given piece of content and its links. We'll get deeper into this in the [next section](#adding-and-modifying-data) and in the page on [entries](/build/entries/). +Holochain has a built-in [**create, read, update, and delete (CRUD)** model](/concepts/6_crud_actions/). Data in the graph database and participants' local state cannot be modified or deleted, so these kinds of mutation are simulated by attaching metadata to existing data. This builds up a graph of the history of a given piece of content. + +We'll get deeper into this in the [next section](#adding-and-modifying-data) and in the page on [entries](/build/entries/). ### Individual state histories as public records All data in an application's database ultimately comes from the peers who participate in storing and serving it. Each piece of data originates in a participant's source chain, which is an [event journal](https://martinfowler.com/eaaDev/EventSourcing.html) that contains all the actions they've authored. These actions describe intentions to add to either the DHT's state or their own state. -Every action becomes part of the shared DHT, but not every entry needs to. The entry content of most system-level actions is private. You can also [mark an application entry type as private](/build/entries/#configuring-an-entry-type), and its content will stay on the participant's device and not get published to the graph. +Every action becomes part of the shared DHT, but not every entry needs to. The entry content of most system-level actions is private. You can also [mark an application entry type as private](/build/entries/#configure-an-entry-type), and its content will stay on the participant's device and not get published to the graph. Because every action has a reference to both its author and its previous action in the author's source chain, each participant's source chain can be considered a linear graph of their authoring history. @@ -121,11 +150,15 @@ The built-in CRUD model is simplistic, collecting all the metadata on an entry o * Although an **entry** can have multiple creation actions attached to it as metadata, the record returned contains the _oldest-timestamped_ entry creation action _that doesn't have a corresponding delete action_. * There's no built-in logic for **updates**, which means that multiple updates can exist on one entry creation action. This creates a branching update model similar to Git and leaves room for you to create your own conflict resolution mechanisms if you need them. Updates aren't retrieved by default; you must retrieve them by [asking for an address' metadata](/build/entries/#all-data-actions-and-links-at-an-address). -* A **delete** applies to an entry creation action, not an entry. An entry is considered live until all its creation actions are deleted, at which point it's fully dead and isn't retrieved when asked for. A dead entry is live once again if a new entry creation action authors it[.](https://youtu.be/d4ftmOI5NnI?si=XeT6OaWg3IN_HYYO&t=42) +* A **delete** applies to an entry creation action, not an entry. An entry is considered live until _all_ of its creation actions are deleted, at which point it's fully dead[.](https://youtu.be/d4ftmOI5NnI?si=XeT6OaWg3IN_HYYO&t=42) A dead entry is live once again if a new entry creation action authors it. * Unlike entries, **links** are completely contained in the action, and are always distinct from each other, even if their base, target, type, and tag are identical. There's no link update action, and a link deletion action marks one link creation action as dead. If these rules don't work for you, you can always [directly access the underlying metadata](/build/entries/#all-data-actions-and-links-at-an-address) and implement your own CRUD model. +#### Deleted/dead data + +Data doesn't ever disappear from the DHT. Instead, deletion actions simply _mark_ an entry or link as dead, which means it won't be retrieved when you ask for it --- unless you [ask for the metadata](/build/entries/#all-data-actions-and-links-at-an-address) at the basis address. + ## Privacy Each **DNA** within a Holochain application has its own network and database, isolated from all other DNAs' networks and their databases. For each participant in a DNA, their source chain is separate from the source chains of all other DNAs they participate in, and the source chains of all other participants in the same DNA. Within a DNA, all shared data can be accessed by any participant, but the only one who can access a participant's private entries is themselves. @@ -136,16 +169,6 @@ A DNA can be **cloned**, creating a separate network, database, and set of sourc The shared DHT and the individual source chains are involved in multiple interrelated graphs --- the source chain contributes to the DHT's graph, and the DHT records source chain history. You can use as little or as much of these graphs as your application needs. -## Further reading - -* Core Concepts: [Source Chain](/concepts/3_source_chain/) -* Core Concepts: [DHT](/concepts/4_dht/) -* Core Concepts: [Links and Anchors](/concepts/5_links_anchors/) -* Core Concepts: [CRUD actions](/concepts/6_crud_actions/) -* Wikipedia: [Graph database](https://en.wikipedia.org/wiki/Graph_database) -* Wikipedia: [Distributed hash table](https://en.wikipedia.org/wiki/Distributed_hash_table) -* Martin Fowler on [Event Sourcing](https://martinfowler.com/eaaDev/EventSourcing.html), the pattern that both source chains and blockchains use to record state changes. - ## Reference * [`holochain_integrity_types::record::Record`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/record/struct.Record.html) @@ -156,8 +179,12 @@ The shared DHT and the individual source chains are involved in multiple interre * [`holochain_integrity_types::prelude::CreateLink`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/prelude/struct.CreateLink.html) * [`holochain_integrity_types::prelude::DeleteLink`](https://docs.rs/holochain_integrity_types/latest/holochain_integrity_types/prelude/struct.DeleteLink.html) -::: topic-list -### In this section {data-no-toc} +## Further reading -* [Entries](/build/entries/) --- creating, reading, updating, and deleting -::: \ No newline at end of file +* Core Concepts: [Source Chain](/concepts/3_source_chain/) +* Core Concepts: [DHT](/concepts/4_dht/) +* Core Concepts: [Links and Anchors](/concepts/5_links_anchors/) +* Core Concepts: [CRUD actions](/concepts/6_crud_actions/) +* Wikipedia: [Graph database](https://en.wikipedia.org/wiki/Graph_database) +* Wikipedia: [Distributed hash table](https://en.wikipedia.org/wiki/Distributed_hash_table) +* Martin Fowler on [Event Sourcing](https://martinfowler.com/eaaDev/EventSourcing.html), the pattern that both source chains and blockchains use to record state changes. \ No newline at end of file diff --git a/src/pages/concepts/6_crud_actions.md b/src/pages/concepts/6_crud_actions.md index 01ca4170f..98297d977 100644 --- a/src/pages/concepts/6_crud_actions.md +++ b/src/pages/concepts/6_crud_actions.md @@ -35,10 +35,10 @@ You might remember from [a few pages back](../3_source_chain/) that we described Here are all the mutation actions an agent can perform: -* A **new-entry action** calls an app or system entry into existence. +* An **entry creation action** calls an app or system entry into existence. * **Create entry** creates a new entry. * **Update entry** also creates a new entry, but marks an existing new-entry action as having an update. -* **Delete entry** marks an existing new-entry action as dead. +* **Delete entry** marks an existing entry creation action as "dead" --- not actually removed from the data store, but not considered 'living' data anymore. (This is because no data ever really disappears from the DHT.) * **Create link** creates a new link. * **Delete link** marks an existing create-link action as dead. diff --git a/src/pages/resources/glossary.md b/src/pages/resources/glossary.md index 0188d8f1d..b89058f6c 100644 --- a/src/pages/resources/glossary.md +++ b/src/pages/resources/glossary.md @@ -13,6 +13,10 @@ A piece of data that represents a [record](#record) on an [agent's](#agent) [sou 1. [DHT address](#dht-address), synonymous with [base](#base) 2. [Transport address](#transport-address) +#### Addressable content + +An individual piece of data in a [content-addressable store](#content-addressable-storage-cas) that can be stored or retrieved by its identifier, usually the hash of the data. + #### Address space The entire range of possible [DHT addresses](#dht-address). This space is circular, meaning the last address is adjacent to the first address. @@ -60,7 +64,7 @@ Any data structure that can only be written to. Once written, that data becomes #### App entry -An entity that holds application data. On the DHT, an app entry is created for every [new entry action](#new-entry-action), and [validation authorities](#validation-authority) who hold the entry also hold the [actions](#action) of all [agents](#agent) who have published that exact same entry as [metadata](#metadata), along with other metadata such as [links](#link). App entries are [deduplicated](#deduplication) but individual agents' writes of those entries are distinguished by their respective actions attached to the entry. +An entity that holds application data. On the DHT, an app entry is created for every [new entry action](#entry-creation-action), and [validation authorities](#validation-authority) who hold the entry also hold the [actions](#action) of all [agents](#agent) who have published that exact same entry as [metadata](#metadata), along with other metadata such as [links](#link). App entries are [deduplicated](#deduplication) but individual agents' writes of those entries are distinguished by their respective actions attached to the entry. #### Application (app) @@ -274,7 +278,7 @@ A simple [coordination protocol](#coordination-protocol) between two or more [ag #### Create-entry action -A [new-entry action](#new-entry-action) that causes an [entry](#entry) to be made available to other [DHT](#distributed-hash-table-dht) members (unless the entry is [private](#private-entry), in which case only a record of its creation is published). +An [entry creation action](#entry-creation-action) that causes an [entry](#entry) to be made available to other [DHT](#distributed-hash-table-dht) members (unless the entry is [private](#private-entry), in which case only a record of its creation is published). #### Create-link action @@ -282,7 +286,7 @@ An [action](#action) that causes a [link](#link) from one piece of [record data] #### Create, read, update, delete (CRUD) -The four main [actions](#action) an application needs to do with data. Even though all data structures in Holochain are [append-only](#append-only), modification and deletion of data can still be simulated by publishing a new action that marks the old data as modified in a [CALM](#consistency-as-logical-monotonicity-calm-theorem) way. [New-entry actions](#new-entry-action) create and/or update entries, while [delete-entry actions](#delete-entry-action) remove them. [Links](#link) can also be created and deleted in a similar way. +The four main [actions](#action) an application needs to do with data. Even though all data structures in Holochain are [append-only](#append-only), modification and deletion of data can still be simulated by publishing a new action that marks the old data as modified in a [CALM](#consistency-as-logical-monotonicity-calm-theorem) way. [Entry creation actions](#entry-creation-action) create and/or update entries, while [delete-entry actions](#delete-entry-action) remove them. [Links](#link) can also be created and deleted in a similar way. #### CRUD action @@ -292,10 +296,10 @@ A [record](#record) that expresses a [CRUD](#create-read-update-delete-crud) ope As no data in a Holochain [DHT](#distributed-hash-table-dht) or [agent's](#agent) [source chain](#source-chain) are ever deleted, existing data must be marked as no longer active. Dead data takes four forms: -1. A [new-entry action](#new-entry-action) action that has been marked as deleted by a [delete-entry action](#delete-entry-action). -2. A [create-link action](#create-link-action) action that has been marked as deleted by a [delete-link action](#delete-link-action). -3. An [entry](#entry) whose new-entry action actions have all been marked as deleted. -4. A [link](#link) whose create-link action actions have all been marked as deleted. +1. An [entry creation action](#entry-creation-action) that has been marked as deleted by a [delete-entry action](#delete-entry-action). +2. A [create-link action](#create-link-action) that has been marked as deleted by a [delete-link action](#delete-link-action). +3. An [entry](#entry) whose creation action(s) have all been marked as deleted. +4. A [link](#link) whose create-link action has been marked as deleted. #### Decentralization @@ -311,7 +315,7 @@ The removal of identical entries in a [CAS](#content-addressable-storage-cas). M #### Delete-entry action -An [action](#action) that causes a [new-entry action](#new-entry-action) to be marked as [dead](#dead-data). If all such actions that caused an [entry](#entry) to be published are marked as dead, the entry itself will also be marked as dead. +An [action](#action) that causes an [entry creation action](#entry-creation-action) to be marked as [dead](#dead-data). If all such actions that caused an [entry](#entry) to be published are marked as dead, the entry itself will also be marked as dead. #### Delete-link action @@ -387,7 +391,11 @@ A channel between two nodes in a public network that allows them to transfer sec #### Entry -A basic unit of application data in a Holochain app. Each entry has its own defined [type](#entry-type). When an [agent](#agent) [commits](#commit) an entry, it is included in an [action](#action) into a [record](#record) that expresses a [new-entry action](#new-entry-action). This data is written to their [source chain](#source-chain) as a record of the action having taken place. An entry can be [public](#public-entry) or [private](#private-entry); if it's public, it's also [published](#publish) to the [DHT](#distributed-hash-table-dht). There are [app entries](#app-entry) whose purpose and structure are defined by the [DNA](#dna) developer, and there are special public or private [system entries](#system-entry) such as an [agent ID entry](#agent-id-entry) and [capability grants](#capability-grant) and [claims](#capability-claim). +A basic unit of application data in a Holochain app. Each entry has its own defined [type](#entry-type). When an [agent](#agent) [commits](#commit) an entry, it is included in an [action](#action) into a [record](#record) that expresses an [entry creation action](#entry-creation-action). This data is written to their [source chain](#source-chain) as a record of the action having taken place. An entry can be [public](#public-entry) or [private](#private-entry); if it's public, it's also [published](#publish) to the [DHT](#distributed-hash-table-dht). There are [app entries](#app-entry) whose purpose and structure are defined by the [DNA](#dna) developer, and there are special public or private [system entries](#system-entry) such as an [agent ID entry](#agent-id-entry) and [capability grants](#capability-grant) and [claims](#capability-claim). + +#### Entry creation action + +Any [action](#action) that writes an [entry](#entry) to the DHT, either a [create-entry](#create-entry-action) or [update-entry](#update-entry-action) action. If the entry's [type](#entry-type) is [public](#public-entry), the entry will be published to the [DHT](#distributed-hash-table-dht) along with its [action](#action). If the entry's type is [private](#private-entry), only the action is published. #### Entry type @@ -647,10 +655,6 @@ In Holochain terms, a collection of [nodes](#node) [gossiping](#gossip) with eac An optional string, specified in a [DNA bundle](#dna-bundle) file or passed at [cell](#cell) [cloning](#cloning) time, that modifies the [DNA's hash](#dna-hash) without modifying any of its behavior. This can be used to create a unique [network](#network) shared by all [agents](#agent) who use the same network seed. Hence, a network seed is considered a [DNA modifier](#dna-modifiers). -#### New-entry action - -Any [action](#action) that produces a new entry, either a [create-entry](#create-entry-action) or [update-entry](#update-entry-action) action. If the entry's [type](#entry-type) is [public](#public-entry), the entry will be published to the [DHT](#distributed-hash-table-dht) along with its [action](#action). If the entry's type is [private](#private-entry), only the action is published. - #### Node An individual [agent](#agent) in a Holochain [network](#network) who has an [agent address](#agent-address) and can be talked to via [gossip](#gossip). @@ -752,7 +756,7 @@ A range of [DHT addresses](#dht-address) for which an [agent](#agent) knows a su --> #### Record -The data structure that holds an [action](#action) in an [agent's](#agent) [source chain](#source-chain). Some records are a combination of [action](#action) and [entry](#entry), such as [new-entry actions](#new-entry-action), while others contain all their data inside the action. +The data structure that holds an [action](#action) in an [agent's](#agent) [source chain](#source-chain). Some records are a combination of [action](#action) and [entry](#entry), such as [entry creation actions](#entry-creation-action), while others contain all their data inside the action. #### Record data @@ -883,7 +887,7 @@ A [capability grant](#capability-grant) that allows any [peer](#peer) or [client #### Update-entry action -A [new-entry action](#new-entry-action) that replaces another new-entry action, essentially allowing the simulated modification of already-written data in a way that allows for multiple branching revision chains. This can be used to modify [public](#public-entry) or [private](#private-entry), [system](#system-entry) or [app](#app-entry) entries. +An [entry creation action](#entry-creation-action) that replaces another entry creation action, essentially allowing the simulated modification of already-written data in a way that allows for multiple branching revision chains. This can be used to modify [public](#public-entry) or [private](#private-entry), [system](#system-entry) or [app](#app-entry) entries. #### Validating DHT diff --git a/src/scss/_base-element-styles.scss b/src/scss/_base-element-styles.scss index d6f9e49ec..70c2cde39 100644 --- a/src/scss/_base-element-styles.scss +++ b/src/scss/_base-element-styles.scss @@ -99,4 +99,34 @@ img { max-width: $p * 10%; } } +} + +table { + width: 100%; + border-collapse: collapse; +} + +th { + background: $cl-light-gray; + color: white; + border-right-color: rgba(white, 0.5); +} + +th, td { + text-align: left; + vertical-align: top; + border-right: 1px solid; + padding: 0 0.5em; + + &:last-child { + border-right: none; + } +} + +td { + border-right-color: rgba(black, 0.5); +} + +tr:nth-child(even) { + background: rgba($cl-gray, 0.5); } \ No newline at end of file