Skip to content

Commit

Permalink
Preview PR pingcap/docs#19727 and this preview is triggered from commit
Browse files Browse the repository at this point in the history
  • Loading branch information
Docsite Preview Bot committed Dec 24, 2024
1 parent d37d60f commit 8e1edbd
Showing 1 changed file with 17 additions and 17 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,10 @@ To address this issue, TiKV introduces [In-Memory Engine (IME)](https://docs.pin
The query latency decreases by 50%, and the throughput increases by 100%.
| Configurations | Duration | Threads | TPS | QPS | Average latency | P95 latency |
| Configuration | Duration (s) | Threads | TPS | QPS | Average latency (ms) | P95 latency (ms) |
| --- | --- | --- | --- | --- | --- | --- |
| Disable IME | 3600 s | 10 | 123.8 | 1498.3 | 80.76 | 207.82 |
| Enable IME | 3600 s | 10 | 238.2 | 2881.5 | 41.99 | 78.60 |
| Disable IME | 3600 | 10 | 123.8 | 1498.3 | 80.76 | 207.82 |
| Enable IME | 3600 | 10 | 238.2 | 2881.5 | 41.99 | 78.60 |
## IO latency jitter
Expand Down Expand Up @@ -87,20 +87,20 @@ The failover time of the IO latency jitter is 30% shorter, and P99/999 latency i
- Test results without IO latency jitter improvement
| Workload | Failover time | QPS drop rate | Maximum latency (P999) | Maximum latency (P99) |
| --- | --- | --- | --- | --- |
| IO delay 1 s lasts for 10 mins | 3 mins | 93% | 4.66 s | 929 ms |
| IO delay 500 ms lasts for 10 mins | 2 mins | 92% | 7.22 s | 894 ms |
| IO delay 100 ms lasts for 10 mins | 3 mins | 80% | 7.53 s | 1.7 s |
| IO delay 50 ms lasts for 10 mins | 3 mins | 53% | 1.36 s | 238 ms |
| IO delay 10 ms lasts for 10 mins | 3 mins | 18% | 69 ms | 25 ms |
| IO delay 5 ms lasts for 10 mins | 2 mins | 29% | 37.9 ms | 10 ms |
| IO delay 2 ms lasts for 10 mins | Almost no effect | 1% | 14 ms | 7.9 ms |
| Workload | Failover time | QPS drop rate | Maximum latency (P999) during failover | Maximum latency (P99) during failover |
| --- | --- | --- | --- | --- |
| IO delay 1 s lasts for 10 mins | 4 mins | 99% | 1 min | 56 s |
| IO delay 500 ms lasts for 10 mins | 4 mins | 99% | 54 s | 7.8 s |
| IO delay 100 ms lasts for 10 mins | Failover not achieved | 99% | 32 s | 26 s |
| IO delay 50 ms lasts for 10 mins | Failover not achieved | 97% | 13.2 s | 6.7 s |
| IO delay 10 ms lasts for 10 mins | Failover not achieved | 94% | 3 s | 1.45 s |
| IO delay 5 ms lasts for 10 mins | Failover not achieved | 81% | 462 ms | 246 ms |
| IO delay 2 ms lasts for 10 mins | Failover not achieved | 38% | 232 ms | 22.9 ms |
- Test results with IO latency jitter improvement
| Workload | Failover time | QPS drop rate | Maximum latency (P999) | Maximum latency (P99) |
| --- | --- | --- | --- | --- |
| Workload | Failover time | QPS drop rate | Maximum latency (P999) during failover | Maximum latency (P99) during failover |
| --- | --- | --- | --- | --- |
| IO delay 1 s lasts for 10 mins | 3 mins | 93% | 4.66 s | 929 ms |
| IO delay 500 ms lasts for 10 mins | 2 mins | 92% | 7.22 s | 894 ms |
| IO delay 100 ms lasts for 10 mins | 3 mins | 80% | 7.53 s | 1.7 s |
Expand All @@ -117,11 +117,11 @@ Large-scale transactions, such as bulk data updates, system migrations, and ETL
- Memory limits: in versions earlier than TiDB v8.1.0, all transaction mutations are held in memory throughout the transaction lifecycle, which strains resources and reduces performance. For operations involving millions of rows, this could lead to excessive memory usage and, in some cases, Out of Memory (OOM) errors when resources are insufficient.
- Performance slowdowns: managing large in-memory buffers relies on red-black trees, which introduces computational overhead. As buffers grew, their operations slowed down due to the *O(N log N)* complexity inherent in these data structures.
- Performance slowdowns: managing large in-memory buffers relies on red-black trees, which introduces computational overhead. As buffers grow, their operations slow down due to the *O(N log N)* complexity inherent in these data structures.
### Solution
These challenges highlighted a clear opportunity to improve scalability, reduce complexity, and enhance reliability. With the rise of modern data workloads, TiDB introduces the [Pipelined DML](https://docs.pingcap.com/tidb/stable/system-variables#tidb_dml_type-new-in-v800) feature, designed to change how large transactions are handled and improve resource utilization and overall performance.
These challenges highlight a clear opportunity to improve scalability, reduce complexity, and enhance reliability. With the rise of modern data workloads, TiDB introduces the [Pipelined DML](https://docs.pingcap.com/tidb/stable/system-variables#tidb_dml_type-new-in-v800) feature, designed to optimize the handling of large transactions, and improve resource utilization and overall performance.
### Test environment
Expand Down Expand Up @@ -172,7 +172,7 @@ To address this issue, TiDB v8.5.0 improves the performance of merging small Reg
#### Scaling out TiKV nodes
- Cluster topology: TiDB (16 vCPU, 32 GiB) \* 1 + TiKV (16 vCPU, 32 GiB) \* 4
- Dataset: TPC-C dataset with 20k warehouses
- Dataset: TPC-C dataset with 20,000 warehouses
- Workload: scaling out TiKV nodes from 4 to 7
### Test results
Expand Down

0 comments on commit 8e1edbd

Please sign in to comment.