Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Relay Mining] Emit events and refactor test helpers #610

Merged
merged 69 commits into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
49c8b34
Checkpoint commit from last night
Olshansk Jun 12, 2024
43b5dee
Merge branch 'main' into issues/542/relay_mining_stress_testing
Olshansk Jun 12, 2024
6791103
Compiled proto update
Olshansk Jun 12, 2024
98a1680
Debugging
Olshansk Jun 12, 2024
3ba4110
Checkpoint commit
Olshansk Jun 13, 2024
f494b38
Unmarshalling seems to work
Olshansk Jun 13, 2024
88e4fe6
Improve comments
Olshansk Jun 13, 2024
61982ee
Self review
Olshansk Jun 13, 2024
36f0aa2
Empty commit
Olshansk Jun 13, 2024
f503c38
Remove log
Olshansk Jun 13, 2024
49a9a2e
Merge branch 'issues/542/fix_session_e2e_test' into issues/542/relay_…
Olshansk Jun 13, 2024
d8b9653
Remove comments
Olshansk Jun 13, 2024
9d67417
Review comments
Olshansk Jun 13, 2024
081bf3b
Review comments
Olshansk Jun 13, 2024
d82649a
Review comments
Olshansk Jun 13, 2024
e8a1cba
Merge with base branch
Olshansk Jun 13, 2024
65540d1
Merge with base branch
Olshansk Jun 14, 2024
8ac5e21
Merge branch 'main' into issues/542/fix_session_e2e_test
Olshansk Jun 14, 2024
469ba9d
Merge branch 'issues/542/fix_session_e2e_test' into issues/542/relay_…
Olshansk Jun 14, 2024
1cb85d6
Add back deleted files
Olshansk Jun 14, 2024
4e0cd68
Integrating with the new SMT updates
Olshansk Jun 14, 2024
2a6a943
Replaced compute units with relay counts
Olshansk Jun 14, 2024
f69a87a
Updated SMT version
Olshansk Jun 14, 2024
a01291b
Merge branch 'main' into issues/542/relay_mining_stress_testing
Olshansk Jun 14, 2024
ae77f86
Added flaky comment
Olshansk Jun 14, 2024
7a00bfd
Update go.mod
Olshansk Jun 14, 2024
7d93164
[RelayMiner] LeanClient (#567)
okdas Jun 15, 2024
446f496
Reply to all review comments
Olshansk Jun 15, 2024
858258b
Merge with main
Olshansk Jun 15, 2024
51dc7f4
Self review
Olshansk Jun 15, 2024
db832e6
Update openapi spec
Olshansk Jun 15, 2024
9716dbc
Added errors to s.Fatalf E2E calls
Olshansk Jun 15, 2024
57482bf
Non functional checkpoint
Olshansk Jun 15, 2024
524b2ca
Checkpoint commit
Olshansk Jun 15, 2024
3df88bb
Build up a foundation
Olshansk Jun 15, 2024
60f35d8
Made progress on E2E test
Olshansk Jun 16, 2024
96f05cc
Reply to all review comments
Olshansk Jun 17, 2024
85b7a9e
Empty commit
Olshansk Jun 17, 2024
18619e2
Merge branch 'issues/542/relay_mining_stress_testing' into issues/542…
Olshansk Jun 17, 2024
dc2109a
Merge branch 'main' into issues/542/relay_mining_events
Olshansk Jun 17, 2024
2fcc981
Got to the point of trying to verify the signature
Olshansk Jun 18, 2024
2f60d49
Moved on to invalid proof from invalid relay request or response
Olshansk Jun 18, 2024
486e906
Small rename
Olshansk Jun 18, 2024
c63a01e
SubmitProof worked
Olshansk Jun 18, 2024
d0af055
A bit of code cleanup
Olshansk Jun 18, 2024
8590be4
Checkpoint commit
Olshansk Jun 19, 2024
c4a03ae
Ready for self review
Olshansk Jun 19, 2024
e504566
Self review
Olshansk Jun 19, 2024
944392f
Merge branch 'main' into issues/542/relay_mining_events
Olshansk Jun 19, 2024
ee35177
Small fix in TestUpdateRelayMiningDifficulty_NewServiceSeenForTheFirs…
Olshansk Jun 19, 2024
b1b60cc
Merge branch 'main' into issues/542/relay_mining_events
Olshansk Jun 19, 2024
f5f219e
Update testutil/events/filter.go
Olshansk Jun 19, 2024
aa87d25
Replied to some PR comments
Olshansk Jun 19, 2024
225687e
Update x/tokenomics/keeper/update_relay_mining_difficulty_test.go
Olshansk Jun 19, 2024
3f89b91
Name getters correctly
Olshansk Jun 19, 2024
f6c7e3f
Moved around some helpers
Olshansk Jun 19, 2024
50ac260
Replied to some PR comments
Olshansk Jun 20, 2024
ca000ba
Replied to some PR comments
Olshansk Jun 20, 2024
95ff2cd
Replied to some PR comments
Olshansk Jun 20, 2024
49fbd78
Fix import
Olshansk Jun 20, 2024
e490908
Merge branch 'main' into issues/542/relay_mining_events
Olshansk Jun 20, 2024
5a39101
Merge with main
Olshansk Jun 20, 2024
7f818b5
Update tests/integration/tokenomics/relay_mining_difficulty_test.go
Olshansk Jun 20, 2024
7cff813
Update tests/integration/tokenomics/relay_mining_difficulty_test.go
Olshansk Jun 20, 2024
add1643
Reply to some of the review comments
Olshansk Jun 20, 2024
9d8c629
Fix emitEvents
Olshansk Jun 20, 2024
5a3f6e3
Empty commit
Olshansk Jun 20, 2024
4869121
Merge branch 'main' into issues/542/relay_mining_events
Olshansk Jun 20, 2024
9fcecac
Increase eventTimeout in E2E tets
Olshansk Jun 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
784 changes: 764 additions & 20 deletions api/poktroll/tokenomics/event.pulsar.go

Large diffs are not rendered by default.

2,474 changes: 585 additions & 1,889 deletions docs/static/openapi.yml

Large diffs are not rendered by default.

533 changes: 267 additions & 266 deletions docusaurus/yarn.lock

Large diffs are not rendered by default.

10 changes: 5 additions & 5 deletions e2e/tests/init_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,7 @@ func (s *suite) getConfigFileContent(amount int64, actorType, serviceId string)
rpc_type: json_rpc`,
amount, serviceId)
default:
s.Fatalf("unknown actor type %s", actorType)
s.Fatalf("ERROR: unknown actor type %s", actorType)
}
fmt.Println(yaml.NormalizeYAMLIndentation(configContent))
return yaml.NormalizeYAMLIndentation(configContent)
Expand Down Expand Up @@ -339,7 +339,7 @@ func (s *suite) TheApplicationIsStakedForService(appName string, serviceId strin
return
}
}
s.Fatalf("application %s is not staked for service %s", appName, serviceId)
s.Fatalf("ERROR: application %s is not staked for service %s", appName, serviceId)
}

func (s *suite) TheSupplierIsStakedForService(supplierName string, serviceId string) {
Expand All @@ -348,7 +348,7 @@ func (s *suite) TheSupplierIsStakedForService(supplierName string, serviceId str
return
}
}
s.Fatalf("supplier %s is not staked for service %s", supplierName, serviceId)
s.Fatalf("ERROR: supplier %s is not staked for service %s", supplierName, serviceId)
}

func (s *suite) TheSessionForApplicationAndServiceContainsTheSupplier(appName string, serviceId string, supplierName string) {
Expand Down Expand Up @@ -377,7 +377,7 @@ func (s *suite) TheSessionForApplicationAndServiceContainsTheSupplier(appName st
return
}
}
s.Fatalf("session for app %s and service %s does not contain supplier %s", appName, serviceId, supplierName)
s.Fatalf("ERROR: session for app %s and service %s does not contain supplier %s", appName, serviceId, supplierName)
}

func (s *suite) TheApplicationSendsTheSupplierARequestForServiceWithPathAndData(appName, supplierName, serviceId, path, requestData string) {
Expand Down Expand Up @@ -572,7 +572,7 @@ func (s *suite) validateAmountChange(prevAmount, currAmount int, expectedAmountC
require.LessOrEqual(s, currAmount, prevAmount, "%s %s expected to have less upokt but actually had more", accName, balanceType)
require.Equal(s, expectedAmountChange, deltaAmount, "%s %s expected) decrease in upokt was incorrect", accName, balanceType)
default:
s.Fatalf("unknown condition %s", condition)
s.Fatalf("ERROR: unknown condition %s", condition)
}

}
Expand Down
12 changes: 6 additions & 6 deletions e2e/tests/parse_params_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ func (s *suite) parseParam(table gocuke.DataTable, rowIdx int) paramAny {
coinValue := cosmostypes.NewCoin(volatile.DenomuPOKT, math.NewInt(coinAmount))
paramValue = &coinValue
default:
s.Fatalf("unexpected param type %q", paramType)
s.Fatalf("ERROR: unexpected param type %q", paramType)
}

return paramAny{
Expand Down Expand Up @@ -116,7 +116,7 @@ func (s *suite) newTokenomicsMsgUpdateParams(params paramsMap) cosmostypes.Msg {
case tokenomicstypes.ParamComputeUnitsToTokensMultiplier:
msgUpdateParams.Params.ComputeUnitsToTokensMultiplier = uint64(paramValue.value.(int64))
default:
s.Fatalf("unexpected %q type param name %q", paramValue.typeStr, paramName)
s.Fatalf("ERROR: unexpected %q type param name %q", paramValue.typeStr, paramName)
}
}
return proto.Message(msgUpdateParams)
Expand All @@ -141,7 +141,7 @@ func (s *suite) newProofMsgUpdateParams(params paramsMap) cosmostypes.Msg {
case prooftypes.ParamProofMissingPenalty:
msgUpdateParams.Params.ProofMissingPenalty = paramValue.value.(*cosmostypes.Coin)
default:
s.Fatalf("unexpected %q type param name %q", paramValue.typeStr, paramName)
s.Fatalf("ERROR: unexpected %q type param name %q", paramValue.typeStr, paramName)
}
}
return proto.Message(msgUpdateParams)
Expand All @@ -168,7 +168,7 @@ func (s *suite) newSharedMsgUpdateParams(params paramsMap) cosmostypes.Msg {
case sharedtypes.ParamProofWindowCloseOffsetBlocks:
msgUpdateParams.Params.ProofWindowCloseOffsetBlocks = uint64(paramValue.value.(int64))
default:
s.Fatalf("unexpected %q type param name %q", paramValue.typeStr, paramName)
s.Fatalf("ERROR: unexpected %q type param name %q", paramValue.typeStr, paramName)
}
}
return proto.Message(msgUpdateParams)
Expand All @@ -188,7 +188,7 @@ func (s *suite) newAppMsgUpdateParams(params paramsMap) cosmostypes.Msg {
case apptypes.ParamMaxDelegatedGateways:
msgUpdateParams.Params.MaxDelegatedGateways = uint64(paramValue.value.(int64))
default:
s.Fatalf("unexpected %q type param name %q", paramValue.typeStr, paramName)
s.Fatalf("ERROR: unexpected %q type param name %q", paramValue.typeStr, paramName)
}
}
return proto.Message(msgUpdateParams)
Expand All @@ -208,7 +208,7 @@ func (s *suite) newServiceMsgUpdateParams(params paramsMap) cosmostypes.Msg {
case servicetypes.ParamAddServiceFee:
msgUpdateParams.Params.AddServiceFee = uint64(paramValue.value.(int64))
default:
s.Fatalf("unexpected %q type param name %q", paramValue.typeStr, paramName)
s.Fatalf("ERROR: unexpected %q type param name %q", paramValue.typeStr, paramName)
}
}
return proto.Message(msgUpdateParams)
Expand Down
2 changes: 1 addition & 1 deletion e2e/tests/reset_params_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ func (s *suite) msgUpdateParamsToDefaultsAny(moduleName string) *codectypes.Any
},
)
default:
s.Fatalf("unknown module name: %s", moduleName)
s.Fatalf("ERROR: unknown module name: %s", moduleName)
}
require.NoError(s, err)

Expand Down
57 changes: 15 additions & 42 deletions e2e/tests/session_steps_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,15 @@ import (

"cosmossdk.io/depinject"
abci "github.com/cometbft/cometbft/abci/types"
cosmostypes "github.com/cosmos/cosmos-sdk/types"
"github.com/stretchr/testify/require"

"github.com/pokt-network/poktroll/pkg/client"
"github.com/pokt-network/poktroll/pkg/client/block"
"github.com/pokt-network/poktroll/pkg/client/events"
"github.com/pokt-network/poktroll/pkg/client/tx"
"github.com/pokt-network/poktroll/pkg/observable/channel"
testutilevents "github.com/pokt-network/poktroll/testutil/events"
"github.com/pokt-network/poktroll/testutil/testclient"
prooftypes "github.com/pokt-network/poktroll/x/proof/types"
tokenomicstypes "github.com/pokt-network/poktroll/x/tokenomics/types"
Expand Down Expand Up @@ -166,13 +168,23 @@ func (s *suite) TheClaimCreatedBySupplierForServiceForApplicationShouldBeSuccess
if event.Type != "poktroll.tokenomics.EventClaimSettled" {
return false
}
claimSettledEvent := s.abciToClaimSettledEvent(event)

// Parse the event
testutilevents.QuoteEventMode(event)
typedEvent, err := cosmostypes.ParseTypedEvent(*event)
require.NoError(s, err)
require.NotNil(s, typedEvent)
claimSettledEvent, ok := typedEvent.(*tokenomicstypes.EventClaimSettled)
require.True(s, ok)

// Assert that the claim was settled for the correct application, supplier, and service.
claim := claimSettledEvent.Claim
require.Equal(s, app.Address, claim.SessionHeader.ApplicationAddress)
require.Equal(s, supplier.Address, claim.SupplierAddress)
require.Equal(s, serviceId, claim.SessionHeader.Service.Id)
require.Greater(s, claimSettledEvent.ComputeUnits, uint64(0), "compute units should be greater than 0")
s.Logf("Claim settled for %d compute units w/ proof requirement: %t\n", claimSettledEvent.ComputeUnits, claimSettledEvent.ProofRequired)

return true
}

Expand Down Expand Up @@ -235,7 +247,7 @@ func (s *suite) waitForTxResultEvent(targetAction string) {

select {
case <-time.After(eventTimeout):
s.Fatalf("timed out waiting for message with action %q", targetAction)
s.Fatalf("ERROR: timed out waiting for message with action %q", targetAction)
case <-ctx.Done():
s.Log("Success; message detected before timeout.")
}
Expand Down Expand Up @@ -280,47 +292,8 @@ func (s *suite) waitForNewBlockEvent(

select {
case <-time.After(eventTimeout):
s.Fatalf("timed out waiting for NewBlock event")
s.Fatalf("ERROR: timed out waiting for NewBlock event")
case <-ctx.Done():
s.Log("Success; message detected before timeout.")
}
}

// abciToClaimSettledEvent converts an abci.Event to a tokenomics.EventClaimSettled
//

func (s *suite) abciToClaimSettledEvent(event *abci.Event) *tokenomicstypes.EventClaimSettled {
var claimSettledEvent tokenomicstypes.EventClaimSettled

// TODO_TECHDEBT: Investigate why `cosmostypes.ParseTypedEvent(*event)` throws
// an error where cosmostypes is imported from "github.com/cosmos/cosmos-sdk/types"
// resulting in the following error:
// 'json: error calling MarshalJSON for type json.RawMessage: invalid character 'E' looking for beginning of value'
// typedEvent, err := cosmostypes.ParseTypedEvent(*event)

for _, attr := range event.Attributes {
switch string(attr.Key) {
case "claim":
var claim prooftypes.Claim
if err := s.cdc.UnmarshalJSON([]byte(attr.Value), &claim); err != nil {
s.Fatalf("Failed to unmarshal claim: %v", err)
}
claimSettledEvent.Claim = &claim
case "compute_units":
value := string(attr.Value)
value = value[1 : len(value)-1] // Remove surrounding quotes
computeUnits, err := strconv.ParseUint(value, 10, 64)
if err != nil {
s.Fatalf("Failed to parse compute_units: %v", err)
}
claimSettledEvent.ComputeUnits = computeUnits
case "proof_required":
proofRequired, err := strconv.ParseBool(string(attr.Value))
if err != nil {
s.Fatalf("Failed to parse proof_required: %v", err)
}
claimSettledEvent.ProofRequired = proofRequired
}
}
return &claimSettledEvent
}
2 changes: 1 addition & 1 deletion pkg/relayer/miner/miner_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ func TestMiner_MinedRelays(t *testing.T) {

// Assert that all minable relay fixtures were published to minedRelays.
actualMinedRelaysMu.Lock()
require.EqualValues(t, expectedMinedRelays, actualMinedRelays)
require.EqualValues(t, expectedMinedRelays, actualMinedRelays, "TODO_FLAKY: Try re-running with 'go test -v -count=1 -run TestMiner_MinedRelays ./pkg/relayer/miner/...'")
actualMinedRelaysMu.Unlock()
}

Expand Down
19 changes: 13 additions & 6 deletions pkg/relayer/session/session_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,23 +25,28 @@ import (
"github.com/pokt-network/poktroll/testutil/testclient/testsupplier"
"github.com/pokt-network/poktroll/testutil/testpolylog"
"github.com/pokt-network/poktroll/testutil/testrelayer"
sessiontypes "github.com/pokt-network/poktroll/x/session/types"
"github.com/pokt-network/poktroll/x/shared"
sharedtypes "github.com/pokt-network/poktroll/x/shared/types"
)

func TestRelayerSessionsManager_Start(t *testing.T) {
const (
sessionStartHeight = 1
sessionEndHeight = 2
)

// TODO_TECHDEBT(#446): Centralize the configuration for the SMT spec.
var (
_, ctx = testpolylog.NewLoggerWithCtx(context.Background(), polyzero.DebugLevel)
spec = smt.NewTrieSpec(sha256.New(), true)
emptyBlockHash = make([]byte, spec.PathHasherSize())
activeSession *sessiontypes.Session
)

activeSession = &sessiontypes.Session{
Header: &sessiontypes.SessionHeader{
SessionStartBlockHeight: 1,
SessionEndBlockHeight: 2,
},
}
sessionHeader := activeSession.GetHeader()

// Set up dependencies.
blocksObs, blockPublishCh := channel.NewReplayObservable[client.Block](ctx, 1)
blockClient := testblock.NewAnyTimesCommittedBlocksSequenceBlockClient(t, emptyBlockHash, blocksObs)
Expand Down Expand Up @@ -94,20 +99,22 @@ func TestRelayerSessionsManager_Start(t *testing.T) {
relayerSessionsManager.Start(ctx)

// Publish a mined relay to the minedRelaysPublishCh to insert into the session tree.
minedRelay := testrelayer.NewMinedRelay(t, sessionStartHeight, sessionEndHeight, supplierAddress)
minedRelay := testrelayer.NewUnsignedMinedRelay(t, activeSession, supplierAddress)
minedRelaysPublishCh <- minedRelay

// Wait a tick to allow the relayer sessions manager to process asynchronously.
// It should have created a session tree for the relay.
time.Sleep(10 * time.Millisecond)

// Publish a block to the blockPublishCh to simulate non-actionable blocks.
sessionStartHeight := sessionHeader.GetSessionStartBlockHeight()
noopBlock := testblock.NewAnyTimesBlock(t, emptyBlockHash, sessionStartHeight)
blockPublishCh <- noopBlock

// Calculate the session grace period end block height to emit that block height
// to the blockPublishCh to trigger claim creation for the session.
sharedParams := sharedtypes.DefaultParams()
sessionEndHeight := sessionHeader.GetSessionEndBlockHeight()
sessionClaimWindowOpenHeight := shared.GetClaimWindowOpenHeight(&sharedParams, sessionEndHeight)

// Publish a block to the blockPublishCh to trigger claim creation for the session.
Expand Down
10 changes: 10 additions & 0 deletions proto/poktroll/tokenomics/event.proto
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,13 @@ message EventClaimSettled {
uint64 compute_units = 2;
bool proof_required = 3;
}

// EventRelayMiningDifficultyUpdated is an event emitted whenever the relay mining difficulty is updated
// for a given service.
message EventRelayMiningDifficultyUpdated {
string service_id = 1;
string prev_target_hash_hex_encoded = 2;
string new_target_hash_hex_encoded = 3;
uint64 prev_num_relays_ema = 4;
uint64 new_num_relays_ema = 5;
}
Loading
Loading