Keystone in CRIB #14326

HenryNguyen5 · 2024-09-04T01:17:37Z

This PR enables keystone in CRIB. In summary, it:

Deploys 2 nodesets, of 5 nodes each
The first nodeset is used to setup a streams trigger DON
The second nodeset is used to setup a workflow DON, with the OCR3 capability and write trigger being local capabilities
The setup includes a workflow that is successfully executed: https://github.com/smartcontractkit/chainlink/pull/14326/files#diff-1d84a52709e4a92e7693dbb5ea24606c6a1d8f28990890bb97bbdcc81852b0beR70

Note that a good portion of this code will be replaced with work done at chainlink/deployments. It's better IMO to get this merged in sooner rather than later to begin integration work with chainlink/deployments ASAP to avoid continuing duplicate implementations of the same provisioning logic.

TODO:

Set mod refs back to their original versions via make gomodslocalupdate
Update chainlink aptos to reflect new repo ref

github-actions · 2024-10-17T06:42:19Z

AER Report: CI Core

aer_workflow , commit , Detect Changes , Scheduled Run Frequency , Clean Go Tidy & Generate , Flakeguard Root Project / Get Tests To Run , GolangCI Lint (core/scripts) , GolangCI Lint (.) , GolangCI Lint (integration-tests) , GolangCI Lint (deployment) , GolangCI Lint (tools/goreleaser-config) , Core Tests (go_core_tests) , Core Tests (go_core_tests_integration) , test-scripts , Core Tests (go_core_ccip_deployment_tests) , Core Tests (go_core_fuzz) , Core Tests (go_core_race_tests) , Flakeguard Deployment Project / Get Tests To Run , Flakeguard Root Project / Run Tests (github.com/smartcontractkit/chainlink/v2/core/cmd, ubuntu-latest) , Flakeguard Deployment Project / Run Tests , Flakeguard Deployment Project / Report , lint , Flakeguard Root Project / Report , SonarQube Scan , Flakey Test Detection

1. File is not `goimports`-ed with -local:

Job ID: Golang Lint (core/scripts)

Source of Error:

Golang Lint (core/scripts)	2024-12-20T19:09:01.9026003Z core/scripts/keystone/src/88_contracts_helpers.go:14: File is not `goimports`-ed with -local github.com/smartcontractkit/chainlink (goimports)
Golang Lint (core/scripts)	2024-12-20T19:09:01.9028115Z 	"github.com/smartcontractkit/chainlink/v2/core/gethwrappers/keystone/generated/capabilities_registry_1_1_0"

Why: The file 88_contracts_helpers.go in the core/scripts/keystone/src directory is not formatted according to the goimports tool with the -local flag set to github.com/smartcontractkit/chainlink.

Suggested fix: Run goimports -local github.com/smartcontractkit/chainlink -w core/scripts/keystone/src/88_contracts_helpers.go to format the file correctly.

2. Test failure in `github.com/smartcontractkit/chainlink/v2/core/capabilities/remote/executable`:

Job ID: Run tests

Source of Error:

Run tests	2024-12-20T19:19:33.8032165Z FAIL
Run tests	2024-12-20T19:19:33.8032593Z FAIL	github.com/smartcontractkit/chainlink/v2/core/capabilities/remote/executable	634.377s

Why: The test suite for github.com/smartcontractkit/chainlink/v2/core/capabilities/remote/executable failed, which caused the overall test run to fail.

Suggested fix: Investigate the specific test cases within core/capabilities/remote/executable to identify the root cause of the failure. Check for any recent changes that might have introduced the issue and ensure that all dependencies and configurations are correct.

3. Encountered test failures:

Job ID: Run tests

Source of Error:

Run tests	2024-12-20T19:19:34.7994372Z go_core_tests exiting with code 1
Run tests	2024-12-20T19:19:34.8009898Z ##[error]Process completed with exit code 1.

Why: The test suite encountered failures, leading to an exit code of 1, which indicates that not all tests passed successfully.

Suggested fix: Review the test logs to identify all failing tests. Address the issues causing the test failures, which may involve fixing bugs, updating test cases, or adjusting configurations.

AER Report: Operator UI CI ran successfully ✅

aer_workflow , commit

core/services/relay/evm/mercury/transmitter.go

HenryNguyen5 · 2024-10-30T22:35:31Z

core/scripts/common/helpers.go

-	PanicErr(err)
-
-	jsonRPCClient, err := rpc.Dial(ethURL)
+	insecureSkipVerify := os.Getenv("INSECURE_SKIP_VERIFY") == "true"


Needed to add this since CRIB can run locally via kind, which uses https but a locally generated cert

HenryNguyen5 · 2024-10-30T22:36:12Z

core/scripts/keystone/main.go

-		src.NewDeployJobSpecsCommand(),
-		src.NewGenerateCribClusterOverridesCommand(),
-		src.NewDeleteJobsCommand(),
+		src.NewProvisionKeystoneCommand(),


For reviewers: This is the main entrypoint, CRIB scripts call into this command only to provision everything. All other commands below are essentially subcmds of this one

HenryNguyen5 · 2024-10-30T22:37:08Z

core/scripts/keystone/main.go

+		src.NewPreprovisionCribCommand(),
+		src.NewPostprovisionCribCommand(),
+		src.NewProvisionCapabilitesRegistryCommand(),
+		src.NewProvisionStreamsTriggerCommand(),
 		src.NewDeployAndInitializeCapabilitiesRegistryCommand(),


@cedric-cordenier Can we remove this? I kept it around because I wasnt sure if the team needs this. NewProvisionCapabilitesRegistryCommand handles provisioning CR for crib

Yes we can remove this if NewProvisionCapabilitesRegistryCommand handles it.

core/scripts/keystone/main.go

HenryNguyen5 · 2024-10-30T22:37:42Z

core/scripts/keystone/src/01_provision_keystone.go

+	return "provision-keystone"
+}
+
+func (g *provisionKeystone) Run(args []string) {


For reviewers: This is the main entrypoint to keystone in CRIB

core/scripts/keystone/src/02_deploy_keystone_workflows.go

cl-sonarqube-production · 2024-10-30T23:36:19Z

Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

core/scripts/go.mod

tools/goreleaser-config/go.mod

… into CRIB-322-keystone-evm-contract-deployment-tooling

github-actions · 2024-11-20T03:37:29Z

Flaky Test Detector for `github.com/smartcontractkit/chainlink` project has failed ❌

Ran new or updated tests between develop and e6c4a3c (CRIB-322-keystone-evm-contract-deployment-tooling).

View Flaky Detector Details | Compare Changes

Flaky Tests

Ran 271 unique tests. Below are the tests identified as flaky, with a pass ratio lower than the 99% threshold:

TestPackage                                        TestName                               PassRatio  RunCount   Skipped
---------                                          ---------                              ---------  ---------  ---------
github.com/smartcontractkit/chainlink/v2/core/cmd  TestShell_ImportExportETHKey_NoChains  67%        3          false
github.com/smartcontractkit/chainlink/v2/core/cmd  TestShell_ListETHKeys_Error            67%        3          false
github.com/smartcontractkit/chainlink/v2/core/cmd  TestShell_CreateETHKey                 67%        3          false

… into CRIB-322-keystone-evm-contract-deployment-tooling

github-actions · 2024-12-12T01:05:12Z

Flakeguard Summary

Ran new or updated tests between develop and 4aea6f7 (CRIB-322-keystone-evm-contract-deployment-tooling).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestShell_ListETHKeys	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/cmd	false	1.173333333s	Unknown

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

github-actions · 2024-12-12T01:23:15Z

Flakeguard Summary

Ran new or updated tests between develop and de1a8f5 (CRIB-322-keystone-evm-contract-deployment-tooling).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestShell_ImportExportETHKey_NoChains	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/cmd	false	1.306666666s	Unknown
TestShell_ListETHKeys	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/cmd	false	1.26s	Unknown

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

core/scripts/keystone/src/99_app.go

… into CRIB-322-keystone-evm-contract-deployment-tooling

github-actions · 2024-12-13T21:20:20Z

Flakeguard Summary

Ran new or updated tests between develop and f4bb112 (CRIB-322-keystone-evm-contract-deployment-tooling).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestShell_ImportExportETHKey_NoChains	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/cmd	false	1.293333333s	Unknown
TestShell_ListETHKeys_Error	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/cmd	false	1.27s	Unknown

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

jmank88 · 2024-12-20T18:55:44Z

tools/goreleaser-config/go.mod

@@ -1,6 +1,6 @@
 module github.com/smartcontractkit/chainlink/tools/goreleaser-config

-go 1.23.0
+go 1.22.8


Is this intentional?

i want to say yes, but i do not remember the reasoning behind it now :/. I could try setting it back and seeing it it breaks any tooling

jmank88 · 2024-12-20T18:57:20Z

core/scripts/keystone/src/99_files.go

+}
+
+func ensureArtefactsDir(artefactsDir string) {
+	_, err := os.Stat(artefactsDir)
 	if err != nil {


Should this be checking for a certain kind of error? https://pkg.go.dev/os#IsNotExist

jmank88 · 2024-12-20T18:59:35Z

core/scripts/keystone/src/01_toolkit_test.go

+	"github.com/stretchr/testify/assert"
+)
+
+func TestMustReadNodesList(t *testing.T) {


Suggested change

func TestMustReadNodesList(t *testing.T) {

func Test_mustReadNodesList(t *testing.T) {

… into CRIB-322-keystone-evm-contract-deployment-tooling

cl-sonarqube-production · 2024-12-20T19:21:13Z

Quality Gate passed

Issues
1 New issue
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

krehermann

This looks good to me

Main request: A README about how to use the tools so that I can push it forward next year

minimal cmds to start, run, stop
cfg or permission required to use it
known limitations

and anything keystone specific that isn't covered in normal crib implementations

justinkaseman

This looks good to me

Main request: A README about how to use the tools so that I can push it forward next year

minimal cmds to start, run, stop
cfg or permission required to use it
known limitations
and anything keystone specific that isn't covered in normal crib implementations

I'll pair with Henry early next week to take notes / record a video on this

PR LGTM at a glance, approving to get out of rebase hell, with further usage next week.

HenryNguyen5 changed the title ~~CRIB 322 keystone evm contract deployment tooling~~ KIAB Streams Trigger Provisioner Sep 4, 2024

HenryNguyen5 force-pushed the CRIB-322-keystone-evm-contract-deployment-tooling branch 3 times, most recently from eaad372 to 7abd46d Compare September 18, 2024 23:59

HenryNguyen5 mentioned this pull request Sep 19, 2024

CRIB 438 add update bridge chainlink cli command #14503

Closed

HenryNguyen5 force-pushed the CRIB-322-keystone-evm-contract-deployment-tooling branch from 6537149 to f0b0226 Compare September 25, 2024 23:00

HenryNguyen5 force-pushed the CRIB-322-keystone-evm-contract-deployment-tooling branch 2 times, most recently from 3a10b2a to 7dfbe68 Compare October 17, 2024 06:21

HenryNguyen5 force-pushed the CRIB-322-keystone-evm-contract-deployment-tooling branch 2 times, most recently from 357886c to 9864099 Compare October 25, 2024 16:28

HenryNguyen5 force-pushed the CRIB-322-keystone-evm-contract-deployment-tooling branch from 7f49528 to b658c21 Compare October 30, 2024 22:33

HenryNguyen5 marked this pull request as ready for review October 30, 2024 22:33

HenryNguyen5 requested review from a team as code owners October 30, 2024 22:33

HenryNguyen5 requested a review from cedric-cordenier October 30, 2024 22:33

HenryNguyen5 commented Oct 30, 2024

View reviewed changes

HenryNguyen5 changed the title ~~KIAB Streams Trigger Provisioner~~ Keystone in CRIB Oct 30, 2024

krehermann previously approved these changes Oct 31, 2024

View reviewed changes

HenryNguyen5 dismissed krehermann’s stale review via e262cf7 November 1, 2024 02:55

HenryNguyen5 requested a review from archseer November 1, 2024 03:02

jmank88 reviewed Nov 1, 2024

View reviewed changes

core/scripts/go.mod Outdated Show resolved Hide resolved

jmank88 reviewed Nov 1, 2024

View reviewed changes

tools/goreleaser-config/go.mod Show resolved Hide resolved

HenryNguyen5 added 3 commits November 7, 2024 16:52

Add node api wrapper for ergonomic cmd usage

48f5f91

Add streams trigger template

11f32e9

Add mock external adapter for v03 mercury

80523fe

HenryNguyen5 added 10 commits November 18, 2024 14:15

Reverse URLs for nodelists

cb46afc

Update snapshots

987b8b5

Remove unneeded gosec ignore

0eb1a29

Merge branch 'develop' of https://github.com/smartcontractkit/chainlink…

f0c9fff

… into CRIB-322-keystone-evm-contract-deployment-tooling

Update gomods

8626cab

Log when we set ocr3 config

376a354

Cleanup argument parsing

a84d5d2

Fix nodes list parsing

83b9d34

Fix lints + address feedback

e27c821

Merge branch 'develop' of https://github.com/smartcontractkit/chainlink…

e6c4a3c

… into CRIB-322-keystone-evm-contract-deployment-tooling

rafaelfelix mentioned this pull request Dec 5, 2024

Fixing cross compilation issue in CRIB #15518

Merged

Merge branch 'develop' of https://github.com/smartcontractkit/chainlink…

4aea6f7

… into CRIB-322-keystone-evm-contract-deployment-tooling

Update gomods to point to this branches pseudo version

de1a8f5

HenryNguyen5 requested a review from a team as a code owner December 12, 2024 01:10

github-advanced-security bot found potential problems Dec 12, 2024

View reviewed changes

core/scripts/keystone/src/99_app.go Dismissed Show dismissed Hide dismissed

core/scripts/keystone/src/99_app.go Dismissed Show dismissed Hide dismissed

core/scripts/keystone/src/99_app.go Dismissed Show dismissed Hide dismissed

HenryNguyen5 added 2 commits December 12, 2024 18:43

Update gomods to point to this branches pseudo version

a05aab5

Merge branch 'develop' of https://github.com/smartcontractkit/chainlink…

f4bb112

… into CRIB-322-keystone-evm-contract-deployment-tooling

jmank88 reviewed Dec 20, 2024

View reviewed changes

HenryNguyen5 added 2 commits December 20, 2024 11:03

Merge branch 'develop' of https://github.com/smartcontractkit/chainlink…

a35f536

… into CRIB-322-keystone-evm-contract-deployment-tooling

Bump wrappers to 1.1.0

2b0a459

krehermann reviewed Dec 20, 2024

View reviewed changes

justinkaseman self-requested a review December 20, 2024 22:37

justinkaseman approved these changes Dec 20, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Keystone in CRIB #14326

Keystone in CRIB #14326

HenryNguyen5 commented Sep 4, 2024 •

edited

Loading

github-actions bot commented Oct 17, 2024 •

edited

Loading

HenryNguyen5 Oct 30, 2024

HenryNguyen5 Oct 30, 2024

HenryNguyen5 Oct 30, 2024

justinkaseman Dec 20, 2024

HenryNguyen5 Oct 30, 2024

cl-sonarqube-production bot commented Oct 30, 2024

github-actions bot commented Nov 20, 2024

github-actions bot commented Dec 12, 2024

github-actions bot commented Dec 12, 2024

github-actions bot commented Dec 13, 2024

jmank88 Dec 20, 2024

HenryNguyen5 Dec 20, 2024 •

edited

Loading

jmank88 Dec 20, 2024

jmank88 Dec 20, 2024

cl-sonarqube-production bot commented Dec 20, 2024

krehermann left a comment •

edited

Loading

justinkaseman left a comment •

edited

Loading

	func TestMustReadNodesList(t *testing.T) {
	func Test_mustReadNodesList(t *testing.T) {

Keystone in CRIB #14326

Are you sure you want to change the base?

Keystone in CRIB #14326

Conversation

HenryNguyen5 commented Sep 4, 2024 • edited Loading

github-actions bot commented Oct 17, 2024 • edited Loading

AER Report: CI Core

1. File is not goimports-ed with -local:

2. Test failure in github.com/smartcontractkit/chainlink/v2/core/capabilities/remote/executable:

3. Encountered test failures:

AER Report: Operator UI CI ran successfully ✅

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cl-sonarqube-production bot commented Oct 30, 2024

Quality Gate failed

github-actions bot commented Nov 20, 2024

Flaky Test Detector for github.com/smartcontractkit/chainlink project has failed ❌

Flaky Tests

github-actions bot commented Dec 12, 2024

Flakeguard Summary

Found Flaky Tests ❌

Artifacts

github-actions bot commented Dec 12, 2024

Flakeguard Summary

Found Flaky Tests ❌

Artifacts

github-actions bot commented Dec 13, 2024

Flakeguard Summary

Found Flaky Tests ❌

Artifacts

Choose a reason for hiding this comment

HenryNguyen5 Dec 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cl-sonarqube-production bot commented Dec 20, 2024

Quality Gate passed

krehermann left a comment • edited Loading

Choose a reason for hiding this comment

justinkaseman left a comment • edited Loading

Choose a reason for hiding this comment

HenryNguyen5 commented Sep 4, 2024 •

edited

Loading

github-actions bot commented Oct 17, 2024 •

edited

Loading

1. File is not `goimports`-ed with -local:

2. Test failure in `github.com/smartcontractkit/chainlink/v2/core/capabilities/remote/executable`:

Flaky Test Detector for `github.com/smartcontractkit/chainlink` project has failed ❌

HenryNguyen5 Dec 20, 2024 •

edited

Loading

krehermann left a comment •

edited

Loading

justinkaseman left a comment •

edited

Loading