Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: LP v2 migrations #1966

Merged
merged 7 commits into from
Sep 9, 2024
Merged

feat: LP v2 migrations #1966

merged 7 commits into from
Sep 9, 2024

Conversation

wischli
Copy link
Contributor

@wischli wischli commented Aug 14, 2024

Description

  • Removes deprecated RelayerList storage
  • Migrates deprecated gateway::DomainRouters storage to axelar_router::Configuration and axelar_router::ChainNameById
  • Centrifuge (v0 -> v3): Migrates deprecated outbound message queues (normal, failed) to corresponding storages in gateway queue
  • Dev (v2 -> v3): Translates queue storage values from v2::GatewayMessage to v3::GatewayMessage

Dev

Migration failed here first such that I removed the old entry and re-added it manually.

024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Pre checks done!
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::nuke] ClearRelayerList: Cleared all RelayerList storage entries
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::nuke] ClearRelayerList Clear result of RelayerList: backend: 0 unique: 0 loops: 0
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Migration done!
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Post checks done!
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::v2_update_message_queue] InitMessageQueue: Pre checks done with 0 items in v2::MessageQueue, 0 in v3::MessageQueue and 0 items in FailedMessageQueue and 0 in v3::FailedMessageQueue!
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::v2_update_message_queue] InitMessageQueue: Migration done with 0 in total!
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::v2_update_message_queue] InitMessageQueue: Post checks done!
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig PRE Migration counts 1 updated domains and 0 removed domains
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig PRE: Checks done with 1 domains!
[2024-09-03T06:51:38Z INFO  frame_support::migrations] 🚚 Pallet "LiquidityPoolsGateway" VersionedMigration migrating storage version from 2 to 3.
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig ON_RUNTIME_UPGRADE: Migration done with 1 updated domains!
[2024-09-03T06:51:38Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig POST: Checks done!
[2024-09-03T06:51:38Z INFO  runtime::frame-support] ⚠️ Ethereum declares internal migrations (which *might* execute). On-chain `StorageVersion(0)` vs current storage version `NoStorageVersionSet`
[2024-09-03T06:51:38Z INFO  runtime::executive] ✅ Entire runtime state decodes without error. 343006 bytes total.

Demo

[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Pre checks done!
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::nuke] ClearRelayerList: Cleared all RelayerList storage entries
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::nuke] ClearRelayerList Clear result of RelayerList: backend: 13 unique: 13 loops: 13
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Migration done!
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Post checks done!
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::v2_update_message_queue] InitMessageQueue: Pre checks done with 0 items in v2::MessageQueue, 0 in v3::MessageQueue and 0 items in FailedMessageQueue and 2 in v3::FailedMessageQueue!
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::v2_update_message_queue] InitMessageQueue: Migration done with 0 in total!
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::v2_update_message_queue] InitMessageQueue: Post checks done!
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig PRE Migration counts 4 updated domains and 0 removed domains
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig PRE: Checks done with 4 domains!
[2024-09-03T06:51:11Z INFO  frame_support::migrations] 🚚 Pallet "LiquidityPoolsGateway" VersionedMigration migrating storage version from 2 to 3.
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig ON_RUNTIME_UPGRADE: Migration done with 4 updated domains!
[2024-09-03T06:51:11Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig POST: Checks done!

Centrifuge Chain

[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Pre checks done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::nuke] ClearRelayerList: Cleared all RelayerList storage entries
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::nuke] ClearRelayerList Clear result of RelayerList: backend: 0 unique: 0 loops: 0
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Migration done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Post checks done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::v0_init_message_queue] InitMessageQueue: Pre checks done with 0 items in OutboundMessageQueue and 0 items in FailedOutboundMessages!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::v0_init_message_queue] InitMessageQueue: Initiating migration of 0 outbound messages
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::v0_init_message_queue] InitMessageQueue: Initiating migration of 0 failed outbound messages
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::v0_init_message_queue] InitMessageQueue: Migration done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::v0_init_message_queue] InitMessageQueue: Post checks done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig PRE Migration counts 4 updated domains and 0 removed domains
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig PRE: Checks done with 4 domains!
[2024-09-03T06:52:12Z INFO  frame_support::migrations] 🚚 Pallet "LiquidityPoolsGateway" VersionedMigration migrating storage version from 0 to 3.
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig ON_RUNTIME_UPGRADE: Migration done with 4 updated domains!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::init_axelar_router] DomainRoutersToAxelarConfig POST: Checks done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Pre checks done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::nuke] ClearRelayerList: Cleared all RelayerList storage entries
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::nuke] ClearRelayerList Clear result of RelayerList: backend: 0 unique: 0 loops: 0
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Migration done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::liquidity_pools_v2::kill_relayer_list] ClearRelayerList: Post checks done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::foreign_investments_v2] ForeignInvestmentsV2: Pre checks done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::foreign_investments_v2] ForeignInvestmentsV2: Removed 3 ForeignInvestmentInfo V1 keys
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::foreign_investments_v2] ForeignInvestmentsV2: Migration done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::foreign_investments_v2] ForeignInvestmentsV2: Post checks done!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "ForeignInvestments" from StorageVersion(1) to StorageVersion(2)
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::permissions_v1] PermssionsV1: Pre checks done!
[2024-09-03T06:52:12Z INFO  frame_support::migrations] 🚚 Pallet "Permissions" VersionedMigration migrating storage version from 0 to 1.
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::permissions_v1] PermssionsV1: Migrated 34 permissions!
[2024-09-03T06:52:12Z INFO  runtime_common::migrations::permissions_v1] PermssionsV1: Post checks done!
[2024-09-03T06:52:12Z INFO  runtime::frame-support] ⚠️ Ethereum declares internal migrations (which *might* execute). On-chain `StorageVersion(0)` vs current storage version `NoStorageVersionSet`
[2024-09-03T06:52:12Z INFO  runtime::executive] ✅ Entire runtime state decodes without error. 6103042 bytes total.

Checklist:

  • I have added Rust doc comments to structs, enums, traits and functions
  • I have made corresponding changes to the documentation
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective or that my feature works

@wischli wischli added the D8-migration Pull request touches storage and needs migration code. label Aug 14, 2024
@wischli wischli self-assigned this Aug 14, 2024
@wischli wischli force-pushed the fix/domain-router-migration branch from 0a07256 to 8ec41e9 Compare August 15, 2024 04:18
Copy link
Contributor Author

@wischli wischli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to merge this PR as DomainRouters has been completely deprecated. Going to use this PR as base for migrating the previous DomainRouters to the new settings storage(s).

@wischli wischli changed the title feat: domain routers migration feat: LP v2 migrations Sep 3, 2024
@wischli wischli marked this pull request as ready for review September 3, 2024 06:52
}
}

pub mod v2_update_message_queue {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Dev and Demo were already upgraded to the intermediary storage version 2 which still supported gateway::DomainRouters storage but removed all routers but the AxelarEvm ones. The corresponding commit hash is 8ec41e9

Therefore, we need to provide migrations from v0 and v2.

Comment on lines +419 to +424
let _ = transactional::with_storage_layer(|| -> DispatchResult {
let (r, w) = Self::migrate_domain_routers()?;
log::info!("{LOG_PREFIX} PRE Migration counts {w:?} updated domains and {} removed domains", r.saturating_sub(w));
writes = w;
Err(DispatchError::Other("Reverting on purpose"))
});
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: There are two pre_upgrade hooks in this PR which utilize a transactional layer to ensure the exact migration call which is applied in the corresponding on_runtime_upgrade does not fail. However, since those migration calls mutate storage, I am reverting the storage changes by failing the transactional layer on purpose.

Copy link

codecov bot commented Sep 3, 2024

Codecov Report

Attention: Patch coverage is 0% with 255 lines in your changes missing coverage. Please review.

Project coverage is 48.28%. Comparing base (d26b99e) to head (bf97ee8).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...untime/common/src/migrations/liquidity_pools_v2.rs 0.00% 247 Missing ⚠️
runtime/common/src/migrations/nuke.rs 0.00% 6 Missing ⚠️
runtime/development/src/lib.rs 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1966      +/-   ##
==========================================
- Coverage   48.99%   48.28%   -0.72%     
==========================================
  Files         183      183              
  Lines       13201    13403     +202     
==========================================
+ Hits         6468     6471       +3     
- Misses       6733     6932     +199     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@wischli wischli requested a review from cdamian September 4, 2024 09:08
Message = GatewayMessage<Message, RouterId>,
>,
{
fn migrate_message_queue(mut reads: u64, mut writes: u64) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't these reads/writes args be references?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They could be yes. But is there any advantage to gain from that? Where do you see an issue with the current style?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed by bf97ee8

Copy link
Collaborator

@mustermeiszer mustermeiszer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Copy link
Contributor

@cdamian cdamian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for tackling these @wischli!

@wischli wischli enabled auto-merge (squash) September 6, 2024 13:24
@wischli
Copy link
Contributor Author

wischli commented Sep 6, 2024

Looks good!

Would you mind approving? Requiring your approval as code owner.

@wischli wischli merged commit 7af4729 into main Sep 9, 2024
12 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
D8-migration Pull request touches storage and needs migration code.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants