Skip to content

Commit

Permalink
doc: add docs for in-memory cache (#9)
Browse files Browse the repository at this point in the history
Signed-off-by: MrCroxx <[email protected]>
  • Loading branch information
MrCroxx authored Sep 16, 2024
1 parent 33f8f3f commit cd18497
Show file tree
Hide file tree
Showing 5 changed files with 128 additions and 53 deletions.
3 changes: 0 additions & 3 deletions docs/01-overview/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ A hybrid cache is a caching system that utilizes both memory and disk storage si

</div>


It is commonly used to extend the insufficient memory cache for the system uses Object Store Service (OSS, e.g. AWS S3) as its primary data storage[^oss-dia] to **improve performance** and **reduce costs**[^risingwave].

## Why we need a hybrid cache?
Expand All @@ -27,7 +26,6 @@ More and more systems are using OSS as their primary data storage. OSS has many

However, there are also downsides with OSS. For example, the latency is high and uncontrollable, and its price increases with each accesses. The downsides will be further amplified in a large working set because of more data exchange between cache and OSS.


<div style="text-align: center;">

![exchange](./assets/exchange.svg)
Expand All @@ -36,7 +34,6 @@ However, there are also downsides with OSS. For example, the latency is high and

With a hybrid cache, the ability to cache the working set can be extended from memory only to memory and disk. This can reduce data exchange between cache and OSS, thereby improving performance and reducing costs.


<div style="text-align: center;">

![exchange hybrid cache](./assets/exchange-hybrid-cache.svg)
Expand Down
118 changes: 118 additions & 0 deletions docs/02-tutorial/01-in-memory-cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
# Setup In-memory Cache

This article will guide you through the process of setting up an in-memory cache.

:::tip for hybrid cache users

The in-memory cache `Cache` is literally an in-memory only cache. Which provides non-overhead in-memory cache implement for users only want to replace their in-memory cache with ***foyer***, and cannot upgrade to a hybrid cache later.

The hybrid cache `HybridCache` also provides a in-memory only mode. If you want to try ***foyer*** out with the in-memory mode and have the needs of a hybrid cache, please use `HybridCache`. See [Tutorial - Hybrid Cache](/docs/tutorial/hybrid-cache).

:::

## 1. Add foyer as a dependency

Add this line to the `[dependencies]` section of your project's `Cargo.toml`.

```toml
foyer = "0.11"
```

If you are using a nightly version of the rust toolchain, the `nightly` feature is needed.

```toml
foyer = { version = "0.11", features = ["nightly"] }
```

## 2. Build a `Cache`

### 2.1 Build with a builder

`Cache`[^cache] can be built from `CacheBuilder`[^cache-builder]. For the default configuration, only `capacity` is required.

```rust
use foyer::{Cache, CacheBuilder};

// ... ...

let cache: Cache<String, String> = CacheBuilder::new(16).build();
```

### 2.2 Count entries by weight

By default, `Cache` counts its usage for eviction by entry count. In the case below, which is 16.

Counting by entry count may be not suitable for all cases. So, ***foyer*** also provides an interface to let the user decide how to count the usage.

```rust
let cache: Cache<String, String> = CacheBuilder::new(10000)
.with_weighter(|key, value| key.len() + value.len())
.build();
```

With `with_weighter`[^with-weighter], you can customize how `Cache` counts its usage. In the case below, the usage is counted by the total character length of the entry.

The weighter can be implemented to count anything. Count, length, memory usage, or anything you want.

### 2.3 Mitigate hot shards

The in-memory cache in ***foyer*** is a sharded cache. Each shard is protected by an individual mutex. Sometimes, one or more shards can be much hotter than others and cause performance regression.

There are two ways to mitigate the hot shards.

1. Scale the shards

Scaling the shard may reduce the possibility to create hot shards. You can scale the shards to a higher number with `with_shards`.[^with-shards]

```rust
let cache: Cache<String, String> = CacheBuilder::new(1024)
.with_shards(64)
.build();
```

2. Use a customized hasher

By default, ***foyer*** uses `ahash`[^ahash] as the hasher to determine which shard a key belongs to. You can use your own hasher with `with_hash_builder`.[^with-hash-builder]

```rust
let cache: Cache<String, String> = CacheBuilder::new(1024)
.with_hash_builder(CustomizedRandomState::default())
.build();
```

### 2.4 More configurations

Please refer to the API document.[^cache-builder]

## 3. Use `Cache` as any other cache library

`Cache` provides familiar interface as any other cache library.

Here is an example.

```rust
use foyer::{Cache, CacheBuilder};

fn main() {
let cache: Cache<String, String> = CacheBuilder::new(16).build();

let entry = cache.insert("hello".to_string(), "world".to_string());
let e = cache.get("hello").unwrap();

assert_eq!(entry.value(), e.value());
}
```

For other interfaces, please refer to the API document.[^cache]

[^cache]: https://docs.rs/foyer/latest/foyer/enum.Cache.html

[^cache-builder]: https://docs.rs/foyer/latest/foyer/struct.CacheBuilder.html

[^with-weighter]: https://docs.rs/foyer/latest/foyer/struct.CacheBuilder.html#method.with_weighter

[^with-shards]: https://docs.rs/foyer/latest/foyer/struct.CacheBuilder.html#method.with_shards

[^ahash]: https://crates.io/crates/ahash

[^with-hash-builder]: https://docs.rs/foyer/latest/foyer/struct.CacheBuilder.html#method.with_hash_builder
50 changes: 0 additions & 50 deletions docs/02-tutorial/01-setup.md

This file was deleted.

7 changes: 7 additions & 0 deletions docs/02-tutorial/02-hybrid-cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Setup Hybrid Cache

This article will guide you through the process of setting up a hybrid cache.

Setting up a hybrid is a little bit complex than setting up an in-memory cache, but not much. Let get through it!

***TBC ... ...***
3 changes: 3 additions & 0 deletions docusaurus.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,9 @@ const config: Config = {

markdown: {
format: 'detect',
remarkRehypeOptions: {
footnoteBackLabel: 'References',
}
},

plugins: [
Expand Down

0 comments on commit cd18497

Please sign in to comment.