Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: update hybrid cache doc part 2 #11

Merged
merged 1 commit into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 84 additions & 3 deletions docs/02-tutorial/02-hybrid-cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ For more details, please refer to the API document.[^hybrid-cache-builder].

:::tip dynamic tail-based tracing config

The tracing config is used for configure the dynamic tail-based tracing. It can also be modified later. To learn more, see [Tutorial - Tail-based Tracing](/docs/tutorial/tail-based-tracing).
The tracing config is used for configure the dynamic tail-based tracing. It can also be modified later. To learn more, see [Tutorial - Setup Monitor System](/docs/tutorial/monitor).

:::

Expand Down Expand Up @@ -78,6 +78,76 @@ RisingWave[^risingwave] supports caching the LSM-tree meta and blocks in both hy

To specify a device for the hybrid cache, just call `with_device_config()`[^with-device-config] and provide the device config.

Currently, the storage of the hybrid cache supports 2 kinds of devices:

1. `DirectFsDevice`[^direct-fs-device]: Setup within a directory of a file system with direct I/O[^direct-io] files.
2. `DirectFileDevice`[^direct-file-device]: Setup with a raw block device (block file) or a direct I/O[^direct-io] file in a file system.

Determine which kind of device to use depends on your needs. If you want to utilize the spare space of an existing file system, `DirectFsDevice` is all you need. If you want to overcome the overhead of the file system, use `DirectFileDevice` on a raw block device. And **NEVER** use a `DirectFileDevice` within a file system. The `inode` latch will make the overhead of the file system layer worse.

:::tip

For more details about the overhead of the file system layer, see [Topic - Overhead of the File System Layer](/docs/topic/overhead-of-the-file-system-layer)

:::

Let's take `DirectFsDevice` as an example:

```rust
let hybrid: HybridCache<u64, String> = HybridCacheBuilder::new()
.with_name("foyer")
.memory(1024)
.storage()
.with_device_config(DirectFsDeviceOptionsBuilder::new("/data/foyer").build())
.build()
.await
.unwrap();
```

This example uses directory `/data/foyer` to store disk cache data using a device options builder. With the default configuration, ***foyer*** will take 80% of the current free space as the disk cache capacity. You can also specify the disk cache capacity and per file size with the builder.

For more details, please refer to the API document.[^direct-fs-device-options-builder] [^direct-file-device-options-builder]

#### 2.3.3 Restrict the throughput

The bandwidth of the disk is much lower than the bandwidth of the memory. To avoid excessive use of the disk bandwidth, it is **HIGHLY RECOMMENDED** to setup the admission picker with a rate limiter.

```rust
let hybrid: HybridCache<u64, String> = HybridCacheBuilder::new()
.with_name("foyer")
.memory(1024)
.storage()
.with_device_config(DirectFsDeviceOptionsBuilder::new("/data/foyer").build())
.with_admission_picker(Arc::new(RateLimitPicker::new(100 * 1024 * 1024)))
.build()
.await
.unwrap();
```

You can also customize the admission picker with your own admission policy. For more details, please refer to the API document.[^admission-picker]

:::tip admission picker

Admission means allowing an entry to be inserted into the disk cache. And a admission picker can be used to customize the admission policy.

For more details, please refer to [Design - Architecture](/docs/design/architecture).

:::

#### 2.3.4 Achieve better performance

The hybrid cache builder also provides a lot of detailed arguments for tuning.

For example:

- `with_indexer_shards()` can be used for mitigating hot shards of the indexer.
- `with_flushers()`, `with_reclaimers()` and `with_recover_concurrency()` can be used to tune the concurrency of the inner components.
- `with_runtime_config()` can be used to enable the dedicated runtime or further runtime splitting.

Tuning the optimized parameters requires an understanding of ***foyer*** interior design and benchmarking with the real workload. For more details, please refer to [Design - Architecture](/docs/design/architecture).

## 3. `HybridCache` Usage

***TBC ... ...***

[^hybrid-cache]: https://docs.rs/foyer/latest/foyer/struct.HybridCache.html
Expand All @@ -88,11 +158,22 @@ To specify a device for the hybrid cache, just call `with_device_config()`[^with

[^storage]: https://docs.rs/foyer/latest/foyer/struct.HybridCacheBuilderPhaseMemory.html#method.storage


[^cache-builder]: https://docs.rs/foyer/latest/foyer/struct.CacheBuilder.html

[^cache]: https://docs.rs/foyer/latest/foyer/enum.Cache.html

[^risingwave]: https://github.com/risingwavelabs/risingwave

[^with-device-config]: https://docs.rs/foyer/latest/foyer/struct.HybridCacheBuilderPhaseStorage.html#method.with_device_config
[^with-device-config]: https://docs.rs/foyer/latest/foyer/struct.HybridCacheBuilderPhaseStorage.html#method.with_device_config

[^direct-fs-device]: https://docs.rs/foyer/latest/foyer/struct.DirectFsDevice.html

[^direct-file-device]: https://docs.rs/foyer/latest/foyer/struct.DirectFileDevice.html

[^direct-io]: https://linux.die.net/HOWTO/SCSI-Generic-HOWTO/dio.html

[^direct-fs-device-options-builder]: https://docs.rs/foyer/latest/foyer/struct.DirectFsDeviceOptionsBuilder.html

[^direct-file-device-options-builder]: https://docs.rs/foyer/latest/foyer/struct.DirectFileDeviceOptionsBuilder.html

[^admission-picker]: https://docs.rs/foyer/latest/foyer/trait.AdmissionPicker.html
3 changes: 3 additions & 0 deletions docs/02-tutorial/03-monitor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Setup Monitor System

***TBC ... ...***
3 changes: 0 additions & 3 deletions docs/02-tutorial/03-tail-based-tracing.md

This file was deleted.

3 changes: 3 additions & 0 deletions docs/03-topic/02-overhead-of-the-file-system-layer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Overhead of the File System Layer

***TBC ... ...***
4 changes: 3 additions & 1 deletion docs/05-design/01-architecture.md
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
# Architecture
# Architecture

***TBC ... ...***
2 changes: 1 addition & 1 deletion docusaurus.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ const config: Config = {
markdown: {
format: 'detect',
remarkRehypeOptions: {
footnoteBackLabel: 'References',
footnoteLabel: 'References',
}
},

Expand Down