Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calibrate single node performance #15100

Merged
merged 2 commits into from
Oct 30, 2024
Merged

Calibrate single node performance #15100

merged 2 commits into from
Oct 30, 2024

Conversation

msmouse
Copy link
Contributor

@msmouse msmouse commented Oct 28, 2024

Description

#14432 made benchmarks emit less events resulting in a dropped calibration quality.

How Has This Been Tested?

Key Areas to Review

Type of Change

  • Tests

Which Components or Systems Does This Change Impact?

  • Developer Infrastructure

Copy link

trunk-io bot commented Oct 28, 2024

⏱️ 2h 19m total CI duration on this PR
Slowest 15 Jobs Cumulative Duration Recent Runs
rust-smoke-tests 49m 🟥🟩
execution-performance / single-node-performance 23m 🟩
rust-move-tests 10m 🟩
rust-move-tests 9m 🟩
rust-move-tests 9m 🟩
rust-cargo-deny 8m 🟩🟩🟩
rust-doc-tests 5m 🟩
check-dynamic-deps 4m 🟩🟩🟩🟩🟩
execution-performance / test-target-determinator 4m 🟩
check 4m 🟩
test-target-determinator 4m 🟩
general-lints 2m 🟩🟩🟩🟩🟩
semgrep/ci 2m 🟩🟩🟩🟩🟩
fetch-last-released-docker-image-tag 2m 🟩
rust-move-tests 1m

settingsfeedbackdocs ⋅ learn more about trunk.io

@msmouse msmouse requested review from igor-aptos, lightmark, a team and zekun000 October 28, 2024 16:59
@msmouse
Copy link
Contributor Author

msmouse commented Oct 28, 2024

#14432 and #15041 baked for too short though, resulting in the values tend to be lower than reality.. maybe we should leave it alone for another day.. @igor-aptos ?

Copy link
Contributor

@lightmark lightmark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks aoshi

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Copy link
Contributor

✅ Forge suite realistic_env_max_load success on c1aa39702b757883311e8158e1c5752a86c3a826

two traffics test: inner traffic : committed: 14290.62 txn/s, latency: 2779.78 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 4100 ms), latency samples: 5433680
two traffics test : committed: 99.97 txn/s, latency: 1490.23 ms, (p50: 1300 ms, p70: 1400, p90: 1600 ms, p99: 5100 ms), latency samples: 1800
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.164, avg: 1.580", "ConsensusProposalToOrdered: max: 0.322, avg: 0.297", "ConsensusOrderedToCommit: max: 0.368, avg: 0.350", "ConsensusProposalToCommit: max: 0.671, avg: 0.647"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.93s no progress at version 2813898 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.22s no progress at version 2813896 (avg 8.22s) [limit 15].
Test Ok

Copy link
Contributor

✅ Forge suite framework_upgrade success on 9c922ebe94f5ff4b58df4617f3ff003e2ce10ccd ==> c1aa39702b757883311e8158e1c5752a86c3a826

Compatibility test results for 9c922ebe94f5ff4b58df4617f3ff003e2ce10ccd ==> c1aa39702b757883311e8158e1c5752a86c3a826 (PR)
Upgrade the nodes to version: c1aa39702b757883311e8158e1c5752a86c3a826
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1144.24 txn/s, submitted: 1147.33 txn/s, failed submission: 3.09 txn/s, expired: 3.09 txn/s, latency: 2611.27 ms, (p50: 2400 ms, p70: 2700, p90: 4200 ms, p99: 6800 ms), latency samples: 103640
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1208.29 txn/s, submitted: 1211.14 txn/s, failed submission: 2.85 txn/s, expired: 2.85 txn/s, latency: 2455.30 ms, (p50: 2100 ms, p70: 2500, p90: 4500 ms, p99: 5800 ms), latency samples: 110240
5. check swarm health
Compatibility test for 9c922ebe94f5ff4b58df4617f3ff003e2ce10ccd ==> c1aa39702b757883311e8158e1c5752a86c3a826 passed
Upgrade the remaining nodes to version: c1aa39702b757883311e8158e1c5752a86c3a826
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1376.01 txn/s, submitted: 1378.01 txn/s, failed submission: 2.00 txn/s, expired: 2.00 txn/s, latency: 2150.14 ms, (p50: 1800 ms, p70: 2100, p90: 3400 ms, p99: 5300 ms), latency samples: 124060
Test Ok

Copy link
Contributor

✅ Forge suite compat success on 9c922ebe94f5ff4b58df4617f3ff003e2ce10ccd ==> c1aa39702b757883311e8158e1c5752a86c3a826

Compatibility test results for 9c922ebe94f5ff4b58df4617f3ff003e2ce10ccd ==> c1aa39702b757883311e8158e1c5752a86c3a826 (PR)
1. Check liveness of validators at old version: 9c922ebe94f5ff4b58df4617f3ff003e2ce10ccd
compatibility::simple-validator-upgrade::liveness-check : committed: 17479.69 txn/s, latency: 1959.92 ms, (p50: 2100 ms, p70: 2100, p90: 2200 ms, p99: 2300 ms), latency samples: 559980
2. Upgrading first Validator to new version: c1aa39702b757883311e8158e1c5752a86c3a826
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6220.04 txn/s, latency: 4433.13 ms, (p50: 5000 ms, p70: 5300, p90: 5600 ms, p99: 5800 ms), latency samples: 113940
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6646.41 txn/s, latency: 4894.96 ms, (p50: 5200 ms, p70: 5300, p90: 6700 ms, p99: 7100 ms), latency samples: 225660
3. Upgrading rest of first batch to new version: c1aa39702b757883311e8158e1c5752a86c3a826
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 6306.88 txn/s, latency: 4437.89 ms, (p50: 5100 ms, p70: 5300, p90: 5600 ms, p99: 5700 ms), latency samples: 113240
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6479.35 txn/s, latency: 4958.12 ms, (p50: 5300 ms, p70: 5400, p90: 6800 ms, p99: 7100 ms), latency samples: 218060
4. upgrading second batch to new version: c1aa39702b757883311e8158e1c5752a86c3a826
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 8750.23 txn/s, latency: 3262.78 ms, (p50: 3600 ms, p70: 3900, p90: 4100 ms, p99: 4300 ms), latency samples: 159440
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 8353.61 txn/s, latency: 3879.09 ms, (p50: 4200 ms, p70: 4300, p90: 4400 ms, p99: 4700 ms), latency samples: 274400
5. check swarm health
Compatibility test for 9c922ebe94f5ff4b58df4617f3ff003e2ce10ccd ==> c1aa39702b757883311e8158e1c5752a86c3a826 passed
Test Ok

@msmouse msmouse merged commit 3ee9895 into main Oct 30, 2024
88 of 92 checks passed
@msmouse msmouse deleted the 1028-alden-cali branch October 30, 2024 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants