Merge branch 'feat/proof-validation' into test/proof-integration

* feat/proof-validation: Offline review of 406 [LocalNet] Fix/localnet regenesis (#424) [CI] Delete validator pod if stuck during E2E test (#415) Fixing reletive link in documentation [Docs] Add code review guidelines & developer tips (#426) [Quick change] Bump default gas limit (#431) chore: Fix unit tests and ring client removal consideration chore: Remove pubkey client chore: Add missing change requests chore: Address review change requests Update pull_request_template.md Update pull_request_template.md [Cleanup] Wrap errors consistently via error types. (#422) [LocalNet] Allow to scale up actors (#414) [Test] Replace time waiting with message submission waiting (#418) Remove accidentally added config files (#420) fix: Ignore externally initiated transactions (#407) [Bug] Address go-routines leak (#410)
pokt-network · Mar 22, 2024 · a262455 · a262455
2 parents a5dca2b + 9179ac1
commit a262455
Show file tree

Hide file tree

Showing 64 changed files with 1,001 additions and 579 deletions.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -10,23 +10,14 @@
 ## Summary
 
 <!-- README: DELETE THIS COMMENT BLOCK after
-      - Provide a quick summary of the changes yourself
-      - Let reviewpad summarize your PR under AI summary
-      - You can leave a `/reviewpad summarize` comment at any time to trigger it manually.
+      - Providing a quick summary of the changes yourself
 -->
 
-### Human Summary
-
-<!-- README: delete this section if your PR is very large --> 
-### AI Summary
-
-reviewpad:summary
-
 ## Issue
 
 <!-- README: DELETE THIS COMMENT BLOCK after:
      - Explain the reasoning for the PR in 1-2 sentences. Adding a screenshot is fair game.
-     - If applicable: specifying the ticket number below if there is a relevant issue. Keep the `-` so the full issue is referenced.
+     - If applicable: specify the ticket number below if there is a relevant issue; _keep the `-` so the full issue is referenced._
 -->
 
 - #{ISSUE_NUMBER}
@@ -43,12 +34,15 @@ Select one or more:
 
 ## Testing
 
-- [ ] **Run all unit tests**: `make go_develop_and_test`
-- [ ] **Run E2E tests locally**: `make test_e2e`
-- [ ] **Run E2E tests on DevNet**: Add the `devnet-test-e2e` label to the PR. This is VERY expensive, only do it after all the reviews are complete.
+- [ ] **Unit Tests**: `make go_develop_and_test`
+- [ ] **LocalNet E2E Tests**: `make test_e2e`
+- [ ] **DevNet E2E Tests**: Add the `devnet-test-e2e` label to the PR. **THIS IS VERY EXPENSIVE**, so only do it after all the reviews are complete.
+- [ ] **Documentation changes**: `make docusaurus_start`
 
 ## Sanity Checklist
 
 - [ ] I have tested my changes using the available tooling
-- [ ] I have performed a self-review of my own code
-- [ ] I have commented my code, updated documentation and left TODOs throughout the codebase
+- [ ] I have commented my code
+- [ ] I have performed a self-review of my own code; both comments & source code
+- [ ] I create and referenced any new tickets, if applicable
+- [ ] I have left TODOs throughout the codebase, if applicable
diff --git a/.github/workflows-helpers/run-e2e-test.sh b/.github/workflows-helpers/run-e2e-test.sh
@@ -1,24 +1,54 @@
+# Enhanced Script for Debugging and Error Handling
+
+# Log environment variables for debugging
+echo "Environment variables:"
+echo "NAMESPACE: ${NAMESPACE}"
+echo "IMAGE_TAG: ${IMAGE_TAG}"
+
 # TODO_TECHDEBT(@okdas): also check readiness of appgate and relayminer to avoid false negatives due to race-conditions
 
-# Check if the pod with the matching image SHA and purpose is ready
-echo "Checking for ready validator pod with image SHA ${IMAGE_TAG}..."
+# Check if the pod with the matching image SHA and purpose is ready or needs recreation
+echo "Checking for ready validator pod with image SHA ${IMAGE_TAG} or pods needing recreation..."
 while :; do
+    # Log the command
+    echo "Running kubectl command to get pods with matching purpose=validator:"
+
     # Get all pods with the matching purpose
-    PODS_JSON=$(kubectl get pods -n ${NAMESPACE} -l pokt.network/purpose=validator -o json)
+    PODS_JSON=$(kubectl get pods -n "${NAMESPACE}" -l pokt.network/purpose=validator -o json)
+
+    # Log the raw output for debugging
+    echo "${PODS_JSON}"
+
+    # Validate JSON output
+    if ! echo "${PODS_JSON}" | jq empty; then
+        echo "Error: kubectl command did not produce valid JSON."
+        exit 1
+    fi
 
     # Check if any pods are running and have the correct image SHA
-    READY_POD=$(echo $PODS_JSON | jq -r ".items[] | select(.status.phase == \"Running\") | select(.spec.containers[].image | contains(\"${IMAGE_TAG}\")) | .metadata.name")
+    READY_POD=$(echo "${PODS_JSON}" | jq -r ".items[] | select(.status.phase == \"Running\") | select(.spec.containers[].image | contains(\"${IMAGE_TAG}\")) | .metadata.name")
+
+    # Check for non-running pods with incorrect image SHA to delete
+    NON_RUNNING_PODS=$(echo "${PODS_JSON}" | jq -r ".items[] | select(.status.phase != \"Running\") | .metadata.name")
+    INCORRECT_POD=$(echo "${NON_RUNNING_PODS}" | jq -r "select(.spec.containers[].image | contains(\"${IMAGE_TAG}\") | not) | .metadata.name")
 
     if [[ -n "${READY_POD}" ]]; then
         echo "Ready pod found: ${READY_POD}"
         break
+    elif [[ -n "${INCORRECT_POD}" ]]; then
+        echo "Non-ready pod with incorrect image found: ${INCORRECT_POD}. Deleting..."
+        kubectl delete pod -n ${NAMESPACE} ${INCORRECT_POD}
+        echo "Pod deleted. StatefulSet will recreate the pod."
+        # Wait for a short duration to allow the StatefulSet to recreate the pod before checking again
+        sleep 10
     else
-        echo "Validator with with an image ${IMAGE_TAG} is not ready yet. Will retry in 10 seconds..."
+        echo "Validator with image ${IMAGE_TAG} is not ready yet and no incorrect pods found. Will retry checking for ready or incorrect pods in 10 seconds..."
         sleep 10
     fi
 done
 
 # Check we can reach the validator endpoint
+echo "Checking HTTP status for the validator endpoint..."
 HTTP_STATUS=$(curl -s -o /dev/null -w '%{http_code}' http://${NAMESPACE}-validator-poktrolld:36657)
 if [[ "${HTTP_STATUS}" -eq 200 ]]; then
     echo "HTTP request to ${NAMESPACE}-validator-poktrolld:36657 returned 200 OK."
@@ -28,21 +58,23 @@ else
 fi
 
 # Create a job to run the e2e tests
+echo "Creating a job to run the e2e tests..."
 envsubst <.github/workflows-helpers/run-e2e-test-job-template.yaml >job.yaml
 kubectl apply -f job.yaml
 
 # Wait for the pod to be created and be in a running state
-echo "Waiting for the pod to be in the running state..."
+echo "Waiting for the e2e test pod to be in the running state..."
 while :; do
     POD_NAME=$(kubectl get pods -n ${NAMESPACE} --selector=job-name=${JOB_NAME} -o jsonpath='{.items[*].metadata.name}')
     [[ -z "${POD_NAME}" ]] && echo "Waiting for pod to be scheduled..." && sleep 5 && continue
     POD_STATUS=$(kubectl get pod ${POD_NAME} -n ${NAMESPACE} -o jsonpath='{.status.phase}')
     [[ "${POD_STATUS}" == "Running" ]] && break
-    echo "Current pod status: ${POD_STATUS}"
+    echo "Current pod status: ${POD_STATUS}. Waiting for 'Running' status..."
     sleep 5
 done
 
 echo "Pod is running. Monitoring logs and status..."
+
 # Stream the pod logs in the background
 kubectl logs -f ${POD_NAME} -n ${NAMESPACE} &
 

diff --git a/Makefile b/Makefile
@@ -242,9 +242,8 @@ localnet_regenesis: check_yq acc_initialize_pubkeys_warn_message ## Regenerate t
 	@echo "Initializing chain..."
 	@set -e ;\
 	ignite chain init --skip-proto ;\
-	mkdir -p $(POKTROLLD_HOME)/config/ ;\
 	cp -r ${HOME}/.poktroll/keyring-test $(POKTROLLD_HOME) ;\
-	cp -r ${HOME}/.poktroll/config/ $(POKTROLLD_HOME)/config/ ;\
+	cp -r ${HOME}/.poktroll/config $(POKTROLLD_HOME)/ ;\
 
 .PHONY: send_relay
 send_relay:
@@ -309,10 +308,7 @@ go_mockgen: ## Use `mockgen` to generate mocks used for testing purposes of all
 	go generate ./x/service/types/
 	go generate ./x/proof/types/
 	go generate ./x/tokenomics/types/
-	go generate ./pkg/client/interface.go
-	go generate ./pkg/miner/interface.go
-	go generate ./pkg/relayer/interface.go
-	go generate ./pkg/crypto/rings/interface.go
+	find . -name interface.go | xargs -I {} go generate {}
 
 .PHONY: go_testgen_fixtures
 go_testgen_fixtures: ## Generate fixture data for unit tests

diff --git a/Tiltfile b/Tiltfile
@@ -10,7 +10,6 @@ localnet_config_path = "localnet_config.yaml"
 localnet_config_defaults = {
     "validator": {"cleanupBeforeEachStart": True},
     "relayminers": {"count": 1},
-    "gateways": {"count": 1},
     "appgateservers": {"count": 1},
     # By default, we use the `helm_repo` function below to point to the remote repository
     # but can update it to the locally cloned repo for testing & development
@@ -98,7 +97,7 @@ k8s_yaml(
     ["localnet/kubernetes/anvil.yaml", "localnet/kubernetes/validator-volume.yaml"]
 )
 
-# Run pocket-specific nodes (validator, relayminers, etc...)
+# Provision validator
 helm_resource(
     "validator",
     chart_prefix + "poktroll-validator",
@@ -111,56 +110,65 @@ helm_resource(
     image_deps=["poktrolld"],
     image_keys=[("image.repository", "image.tag")],
 )
-helm_resource(
-    "relayminers",
-    chart_prefix + "relayminer",
-    flags=[
-        "--values=./localnet/kubernetes/values-common.yaml",
-        "--values=./localnet/kubernetes/values-relayminer.yaml",
-        "--set=replicaCount=" + str(localnet_config["relayminers"]["count"]),
-    ],
-    image_deps=["poktrolld"],
-    image_keys=[("image.repository", "image.tag")],
-)
-if localnet_config["appgateservers"]["count"] > 0:
+
+# Provision RelayMiners
+actor_number = 0
+for x in range(localnet_config["relayminers"]["count"]):
+    actor_number = actor_number + 1
+    helm_resource(
+        "relayminer" + str(actor_number),
+        chart_prefix + "relayminer",
+        flags=[
+            "--values=./localnet/kubernetes/values-common.yaml",
+            "--values=./localnet/kubernetes/values-relayminer-common.yaml",
+            "--values=./localnet/kubernetes/values-relayminer-"+str(actor_number)+".yaml",
+        ],
+        image_deps=["poktrolld"],
+        image_keys=[("image.repository", "image.tag")],
+    )
+    k8s_resource(
+        "relayminer"  + str(actor_number),
+        labels=["supplier_nodes"],
+        resource_deps=["validator"],
+        port_forwards=[
+            str(8084+actor_number)+":8545", # relayminer1 - exposes 8545, relayminer2 exposes 8546, etc.
+            str(40044+actor_number)+":40005", # DLV port. relayminer1 - exposes 40045, relayminer2 exposes 40046, etc.
+            # Run `curl localhost:PORT` to see the current snapshot of relayminer metrics.
+            str(9069+actor_number)+":9090", # Relayminer metrics port. relayminer1 - exposes 9070, relayminer2 exposes 9071, etc.
+        ],
+    )
+
+# Provision AppGate Servers
+actor_number = 0
+for x in range(localnet_config["appgateservers"]["count"]):
+    actor_number = actor_number + 1
     helm_resource(
-        "appgateservers",
+        "appgateserver" + str(actor_number),
         chart_prefix + "appgate-server",
         flags=[
             "--values=./localnet/kubernetes/values-common.yaml",
             "--values=./localnet/kubernetes/values-appgateserver.yaml",
-            "--set=replicaCount=" + str(localnet_config["appgateservers"]["count"]),
+            "--set=config.signing_key=app" + str(actor_number),
         ],
         image_deps=["poktrolld"],
         image_keys=[("image.repository", "image.tag")],
     )
-
-k8s_resource(
-    "validator",
-    labels=["pocket_network"],
-    port_forwards=["36657", "36658", "40004"],
-)
-k8s_resource(
-    "relayminers",
-    labels=["supplier_nodes"],
-    resource_deps=["validator"],
-    port_forwards=[
-        "8545",
-        "40005",
-        # Run `curl localhost:9094` to see the current snapshot of relayminer metrics.
-        "9094:9090",
-    ],
-)
-if localnet_config["appgateservers"]["count"] > 0:
     k8s_resource(
-        "appgateservers",
+        "appgateserver" + str(actor_number),
         labels=["supplier_nodes"],
         resource_deps=["validator"],
         port_forwards=[
-            "42069",
-            "40006",
-            # Run `curl localhost:9093` to see the current snapshot of appgateserver metrics.
-            "9093:9090",
+            str(42068+actor_number)+":42069", # appgateserver1 - exposes 42069, appgateserver2 exposes 42070, etc.
+            str(40054+actor_number)+":40006", # DLV port. appgateserver1 - exposes 40055, appgateserver2 exposes 40056, etc.
+            # Run `curl localhost:PORT` to see the current snapshot of appgateserver metrics.
+            str(9079+actor_number)+":9090", # appgateserver metrics port. appgateserver1 - exposes 9080, appgateserver2 exposes 9081, etc.
         ],
     )
+
+k8s_resource(
+    "validator",
+    labels=["pocket_network"],
+    port_forwards=["36657", "36658", "40004"],
+)
+
 k8s_resource("anvil", labels=["data_nodes"], port_forwards=["8547"])
diff --git a/config.yml b/config.yml
@@ -111,7 +111,7 @@ genesis:
             - endpoints:
                 - configs: []
                   rpc_type: JSON_RPC
-                  url: http://relayminers:8545
+                  url: http://relayminer1:8545
               service:
                 id: anvil
                 name: ""

diff --git a/docusaurus/docs/README.md b/docusaurus/docs/README.md
@@ -63,7 +63,7 @@ roadmap changelog [here](./roadmap/roadmap_changelog.md).
 
 ### Quickstart
 
-The best way to get involved is by following the instructions [here](./quickstart.md).
+The best way to get involved is by following the instructions [here](./developer_guide/quickstart.md).
 
 ### Godoc
 

diff --git a/docusaurus/docs/configs/supplier_staking_config.md b/docusaurus/docs/configs/supplier_staking_config.md
@@ -104,8 +104,10 @@ to the Pocket Network. Each `endpoint` object consists of an `url` and a `rpc_ty
 
 _`Required`_
 
-`url` is a string formatted URL that defines the endpoint that MUST be reachable by
-`Gateways` and `Applications` to send `RelayRequests` to.
+The `url` defines the endpoint for sending `RelayRequests` from the Pocket Network's `Gateways` and `Applications`. Typically, this endpoint is provided by a RelayMiner, which acts as a proxy. This setup means that the specified URL directs to the RelayMiner's endpoint, which in turn, routes the requests to the designated service node.
+
+- **Example**: `https://etherium-relayminer1.relayminers.com:443` demonstrates how a RelayMiner endpoint, typically offered by the RelayMiner, proxies requests to an Etherium backend data node.
+
 
 ##### `rpc_type`
 

diff --git a/docusaurus/docs/developer_guide/_category_.json b/docusaurus/docs/developer_guide/_category_.json
@@ -0,0 +1,8 @@
+{
+  "label": "Developer Guide",
+  "position": 2,
+  "link": {
+    "type": "generated-index",
+    "description": "Documentation related to onboarding as a developer to the Pocket Network protocol."
+  }
+}