Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(x/auth): Fix system test #20531

Merged
merged 40 commits into from
Jun 12, 2024
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
214605b
change initial version to 0
sontrinh16 May 31, 2024
aefee6a
Revert "change initial version to 0"
sontrinh16 Jun 1, 2024
495ab79
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk
sontrinh16 Jun 1, 2024
f761eb1
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk
sontrinh16 Jun 3, 2024
13cd0df
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk
sontrinh16 Jun 3, 2024
9c3170a
fix system test
sontrinh16 Jun 3, 2024
c32693e
Merge branch 'main' into son/quick_fix
sontrinh16 Jun 3, 2024
af58812
move acc num sync to upgarde handler
sontrinh16 Jun 4, 2024
3b33ae9
revert main logic for now
sontrinh16 Jun 4, 2024
e6afd5e
Merge branch 'son/quick_fix' of https://github.com/cosmos/cosmos-sdk …
sontrinh16 Jun 4, 2024
521c566
minor
sontrinh16 Jun 4, 2024
c1a08da
fix accounts init genesis problem and fix system and sim test
sontrinh16 Jun 4, 2024
dfebc64
go mod tidy
sontrinh16 Jun 4, 2024
d1244ae
minor
sontrinh16 Jun 4, 2024
94e9b2b
Merge branch 'main' into son/quick_fix
sontrinh16 Jun 4, 2024
1d9cfcf
address comments
sontrinh16 Jun 5, 2024
d4d9a11
Merge branch 'son/quick_fix' of https://github.com/cosmos/cosmos-sdk …
sontrinh16 Jun 5, 2024
ba123e6
Merge branch 'main' into son/quick_fix
sontrinh16 Jun 9, 2024
c43114f
make deprecate
sontrinh16 Jun 9, 2024
e5206f6
add test case
sontrinh16 Jun 10, 2024
179641d
address comments
sontrinh16 Jun 10, 2024
663ef59
update upgrade doc
sontrinh16 Jun 10, 2024
610ef6f
minor
sontrinh16 Jun 10, 2024
7388cbd
add test for upgrade logic
sontrinh16 Jun 10, 2024
3441aae
minor
sontrinh16 Jun 10, 2024
e68685a
duplicate comment
sontrinh16 Jun 10, 2024
51742cc
Merge branch 'main' into son/quick_fix
sontrinh16 Jun 10, 2024
f60aa6b
Update simapp/upgrades_test.go
sontrinh16 Jun 10, 2024
49ef1e3
fix simapp test
sontrinh16 Jun 10, 2024
5f9818b
Merge branch 'son/quick_fix' of https://github.com/cosmos/cosmos-sdk …
sontrinh16 Jun 10, 2024
053ae26
address comment
sontrinh16 Jun 11, 2024
eae8154
changelog, add detail err log
sontrinh16 Jun 11, 2024
3ef3407
address comments
sontrinh16 Jun 11, 2024
c9aa0f2
lint
sontrinh16 Jun 11, 2024
6cf6894
fix upgrading doc
sontrinh16 Jun 11, 2024
c22d05c
Merge branch 'main' into son/quick_fix
sontrinh16 Jun 11, 2024
75d5bab
more lint
sontrinh16 Jun 12, 2024
c2c2b1c
fix conflict
sontrinh16 Jun 12, 2024
69d6738
lint
sontrinh16 Jun 12, 2024
1d1b745
lint more
sontrinh16 Jun 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions UPGRADING.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,17 @@ Most of Cosmos SDK modules have migrated to [collections](https://docs.cosmos.ne
Many functions have been removed due to this changes as the API can be smaller thanks to collections.
For modules that have migrated, verify you are checking against `collections.ErrNotFound` when applicable.

#### `x/accounts`

Accounts's AccountNumber will be used as a global account number tracking replacing Auth legacy AccountNumber. Must set accounts's AccountNumber with auth's AccountNumber value in upgrade handler. This is done through auth keeper MigrateAccountNumber function.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add that this should be done in the ugprade handler of an app when wanting using x/accounts.

Can we have this as a diff of upgrade.go like we did for protocolpool?


```go
err := app.AuthKeeper.MigrateAccountNumber(ctx)
if err != nil {
return nil, err
}
tac0turtle marked this conversation as resolved.
Show resolved Hide resolved
```

#### `x/auth`

Auth was spun out into its own `go.mod`. To import it use `cosmossdk.io/x/auth`
Expand Down
6 changes: 6 additions & 0 deletions simapp/upgrades.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ func (app SimApp) RegisterUpgradeHandlers() {
app.UpgradeKeeper.SetUpgradeHandler(
UpgradeName,
func(ctx context.Context, _ upgradetypes.Plan, fromVM appmodule.VersionMap) (appmodule.VersionMap, error) {
// sync accounts and auth module account number
sontrinh16 marked this conversation as resolved.
Show resolved Hide resolved
err := app.AuthKeeper.MigrateAccountNumberUnsafe(ctx)
if err != nil {
return nil, err
}

sontrinh16 marked this conversation as resolved.
Show resolved Hide resolved
return app.ModuleManager.RunMigrations(ctx, app.Configurator(), fromVM)
},
)
Expand Down
51 changes: 51 additions & 0 deletions simapp/upgrades_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package simapp

import (
"testing"

"cosmossdk.io/collections"
authtypes "cosmossdk.io/x/auth/types"
cmtproto "github.com/cometbft/cometbft/api/cometbft/types/v1"
"github.com/stretchr/testify/require"
)

// TestSyncAccountNumber tests if accounts module account number is set correctly with the value get from auth.
// Also check if the store entry for auth GlobalAccountNumberKey is successfully deleted.
func TestSyncAccountNumber(t *testing.T) {
app := Setup(t, true)
ctx := app.NewUncachedContext(true, cmtproto.Header{})

bytesKey := authtypes.GlobalAccountNumberKey
store := app.AuthKeeper.KVStoreService.OpenKVStore(ctx)

// initially there is no value set yet
v, err := store.Get(bytesKey)
require.NoError(t, err)
require.Nil(t, v)

// set value for legacy account number
v, err = collections.Uint64Value.Encode(10)
require.NoError(t, err)
err = store.Set(bytesKey, v)
require.NoError(t, err)

// make sure value are updated
v, err = store.Get(bytesKey)
require.NoError(t, err)
require.NotEmpty(t, v)
num, err := collections.Uint64Value.Decode(v)
require.NoError(t, err)
require.Equal(t, uint64(10), num)

app.AuthKeeper.MigrateAccountNumberUnsafe(ctx)
sontrinh16 marked this conversation as resolved.
Show resolved Hide resolved

// make sure the DB entry for this key is deleted
v, err = store.Get(bytesKey)
require.NoError(t, err)
require.Nil(t, v)

// check if accounts's account number is updated
currentNum, err := app.AccountsKeeper.AccountNumber.Peek(ctx)
require.NoError(t, err)
require.Equal(t, uint64(10), currentNum)
}
21 changes: 15 additions & 6 deletions x/accounts/genesis.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,19 +64,28 @@ func (k Keeper) exportAccount(ctx context.Context, addr []byte, accType string,
}

func (k Keeper) ImportState(ctx context.Context, genState *v1.GenesisState) error {
err := k.AccountNumber.Set(ctx, genState.AccountNumber)
if err != nil {
return err
}

var largestNum *uint64
var err error
// import accounts
for _, acc := range genState.Accounts {
err = k.importAccount(ctx, acc)
if err != nil {
return fmt.Errorf("%w: %s", err, acc.Address)
}

accNum := acc.AccountNumber

if largestNum == nil || *largestNum < accNum {
sontrinh16 marked this conversation as resolved.
Show resolved Hide resolved
largestNum = &accNum
}
}
return nil

if largestNum != nil {
// set the account number to the highest account number to avoid duplicate account number
err = k.AccountNumber.Set(ctx, *largestNum)
}

return err
}

func (k Keeper) importAccount(ctx context.Context, acc *v1.GenesisAccount) error {
Expand Down
17 changes: 17 additions & 0 deletions x/accounts/genesis_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,23 @@ func TestGenesis(t *testing.T) {
resp, err = k.Query(ctx, addr2, &types.DoubleValue{})
require.NoError(t, err)
require.Equal(t, &types.UInt64Value{Value: 20}, resp)

// reset state
k, ctx = newKeeper(t, func(deps implementation.Dependencies) (string, implementation.Account, error) {
acc, err := NewTestAccount(deps)
return testAccountType, acc, err
})

// modify the accounts account number
state.Accounts[0].AccountNumber = 99

err = k.ImportState(ctx, state)
require.NoError(t, err)

currentAccNum, err := k.AccountNumber.Peek(ctx)
require.NoError(t, err)
// AccountNumber should be set to the highest account number in the genesis state
require.Equal(t, uint64(99), currentAccNum)
Comment on lines +54 to +70
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor the test to ensure it covers scenarios with multiple account numbers.

The current test modifies and checks a single account number. Considering the changes in the accounts module to handle multiple account numbers, it would be beneficial to extend this test to cover scenarios where multiple account numbers are modified and verified. This would ensure more robust testing of the new logic introduced in the accounts module.

}

func TestImportAccountError(t *testing.T) {
Expand Down
36 changes: 36 additions & 0 deletions x/auth/keeper/keeper.go
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,10 @@ type AccountKeeper struct {
Schema collections.Schema
Params collections.Item[types.Params]

// only use for upgrade handler
//
// Deprecated: move to accounts module accountNumber
accountNumber collections.Sequence
sontrinh16 marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure proper deprecation of accountNumber.

The accountNumber field is marked as deprecated but is still being used in the RemoveLegacyAccountNumber and MigrateAccountNumberUnsafe methods. Consider removing or refactoring these usages to align with the deprecation strategy.

// Accounts key: AccAddr | value: AccountI | index: AccountsIndex
Accounts *collections.IndexedMap[sdk.AccAddress, sdk.AccountI, AccountsIndexes]
}
Expand Down Expand Up @@ -135,6 +139,7 @@ func NewAccountKeeper(
permAddrs: permAddrs,
authority: authority,
Params: collections.NewItem(sb, types.ParamsKey, "params", codec.CollValue[types.Params](cdc)),
accountNumber: collections.NewSequence(sb, types.GlobalAccountNumberKey, "account_number"),
Accounts: collections.NewIndexedMap(sb, types.AddressStoreKeyPrefix, "accounts", sdk.AccAddressKey, codec.CollInterfaceValue[sdk.AccountI](cdc), NewAccountIndexes(sb)),
}
schema, err := sb.Build()
Expand All @@ -145,6 +150,22 @@ func NewAccountKeeper(
return ak
}

// RemoveLegacyAccountNumber is used for migration purpose only. It deletes the sequence in the DB
// and returns the last value used on success.
// Deprecated
func (ak AccountKeeper) RemoveLegacyAccountNumber(ctx context.Context) (uint64, error) {
sontrinh16 marked this conversation as resolved.
Show resolved Hide resolved
accNum, err := ak.accountNumber.Peek(ctx)
if err != nil {
return 0, err
}

// Delete DB entry for legacy account number
store := ak.KVStoreService.OpenKVStore(ctx)
err = store.Delete(types.GlobalAccountNumberKey.Bytes())

return accNum, err
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarify the deprecation and removal process for RemoveLegacyAccountNumber.

The RemoveLegacyAccountNumber method is marked as deprecated, but it's unclear when and how it will be removed from the codebase. It would be beneficial to outline a clear deprecation and removal strategy in the documentation to avoid confusion and ensure that other modules do not inadvertently depend on this deprecated functionality.


// GetAuthority returns the x/auth module's authority.
func (ak AccountKeeper) GetAuthority() string {
return ak.authority
Expand Down Expand Up @@ -317,3 +338,18 @@ func (ak AccountKeeper) NonAtomicMsgsExec(ctx context.Context, signer sdk.AccAdd

return msgResponses, nil
}

// MigrateAccountNumberUnsafe migrates auth's account number to accounts's account number
// and delete store entry for auth legacy GlobalAccountNumberKey.
//
// Should only use in an upgrade handler for migrate process.
func (ak AccountKeeper) MigrateAccountNumberUnsafe(ctx context.Context) error {
sontrinh16 marked this conversation as resolved.
Show resolved Hide resolved
currentAccNum, err := ak.RemoveLegacyAccountNumber(ctx)
if err != nil {
return err
}

err = ak.AccountsModKeeper.InitAccountNumberSeqUnsafe(ctx, currentAccNum)

return err
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review the usage of MigrateAccountNumberUnsafe.

The MigrateAccountNumberUnsafe method is critical for the migration process but lacks detailed error handling and logging. Consider adding more robust error handling and detailed logging to trace the migration steps and potential issues.

21 changes: 2 additions & 19 deletions x/auth/keeper/migrations.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ package keeper
import (
"context"

"cosmossdk.io/collections"
v5 "cosmossdk.io/x/auth/migrations/v5"
"cosmossdk.io/x/auth/types"

Expand All @@ -13,16 +12,11 @@ import (
// Migrator is a struct for handling in-place store migrations.
type Migrator struct {
keeper AccountKeeper
// accNum is use in v4 to v5 and v5 to v6 migration
accNum collections.Sequence
}

// NewMigrator returns a new Migrator.
func NewMigrator(keeper AccountKeeper) Migrator {
sb := collections.NewSchemaBuilder(keeper.Environment.KVStoreService)
accNumSeq := collections.NewSequence(sb, types.GlobalAccountNumberKey, "account_number")

return Migrator{keeper: keeper, accNum: accNumSeq}
return Migrator{keeper: keeper}
}

// Migrate1to2 migrates from version 1 to 2.
Expand All @@ -48,18 +42,7 @@ func (m Migrator) Migrate3to4(ctx context.Context) error {
// It migrates the GlobalAccountNumber from being a protobuf defined value to a
// big-endian encoded uint64, it also migrates it to use a more canonical prefix.
func (m Migrator) Migrate4To5(ctx context.Context) error {
return v5.Migrate(ctx, m.keeper.KVStoreService, m.accNum)
}

// Migrate5To6 migrates the x/auth module state from the consensus version 5 to 6.
// It migrates the GlobalAccountNumber from x/auth to x/accounts .
func (m Migrator) Migrate5To6(ctx context.Context) error {
currentAccNum, err := m.accNum.Peek(ctx)
if err != nil {
return err
}

return m.keeper.AccountsModKeeper.InitAccountNumberSeqUnsafe(ctx, currentAccNum)
return v5.Migrate(ctx, m.keeper.KVStoreService, m.keeper.accountNumber)
}

// V45SetAccount implements V45_SetAccount
Expand Down
12 changes: 0 additions & 12 deletions x/auth/migrations/v6/migrate.go

This file was deleted.

5 changes: 1 addition & 4 deletions x/auth/module.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import (

// ConsensusVersion defines the current x/auth module consensus version.
const (
ConsensusVersion = 6
ConsensusVersion = 5
GovModuleName = "gov"
)

Expand Down Expand Up @@ -113,9 +113,6 @@ func (am AppModule) RegisterMigrations(mr appmodule.MigrationRegistrar) error {
if err := mr.Register(types.ModuleName, 4, m.Migrate4To5); err != nil {
return fmt.Errorf("failed to migrate x/%s from version 4 to 5: %w", types.ModuleName, err)
}
if err := mr.Register(types.ModuleName, 5, m.Migrate5To6); err != nil {
return fmt.Errorf("failed to migrate x/%s from version 5 to 6: %w", types.ModuleName, err)
}

return nil
}
Expand Down
Loading