Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(systemtests): fix gRPC tests for v1 & v2 #22774

Merged
merged 6 commits into from
Dec 6, 2024
Merged

Conversation

julienrbrt
Copy link
Member

@julienrbrt julienrbrt commented Dec 5, 2024

Description

ref: #22753

Fixes TestAuthzGRPCQueries & TestBankGRPCQueries

The fix updates the systemtests framework to support error codes greater than equal.
We've decided to not keep the same behavior in v1 & v2, as the behavior in v2 is supperior.

v1 returns 500 for each failing queries, while v2 returns bad request when it is an user error.


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title, you can find examples of the prefixes below:
  • confirmed ! in the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • reviewed "Files changed" and left comments if necessary
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • updated the relevant documentation or specification, including comments for documenting Go code
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic, API design and naming, documentation is accurate, tests and test coverage

Summary by CodeRabbit

  • New Features

    • Enhanced error messages for state retrieval failures in the app manager.
    • Added support for "greater than or equal" in REST test cases.
    • Introduced a new field for expected HTTP status codes in REST test cases.
  • Bug Fixes

    • Improved error handling for insufficient funds and unauthorized signatures in bank transactions.
    • Updated expected HTTP status codes for various test scenarios.
  • Documentation

    • Updated changelog for version v1.0.0-rc.3.
  • Refactor

    • Restructured test cases for clarity and consistency across various test files.

Copy link
Contributor

coderabbitai bot commented Dec 5, 2024

📝 Walkthrough
📝 Walkthrough

Walkthrough

The changes in this pull request primarily involve enhancements to error handling and testing frameworks across multiple files. The Query method in the appManager struct now returns a more descriptive error message. The systemtests directory sees updates to the changelog and REST test cases, introducing new functionality for expected HTTP status codes. Additionally, various test files have been refactored for clarity and consistency in structure, particularly in authorization and bank transaction tests.

Changes

File Change Summary
server/v2/appmanager/appmanager.go Updated error handling in Query method to return a more descriptive message for state retrieval failures.
systemtests/CHANGELOG.md Added entry for version v1.0.0-rc.3, documenting the addition of "greater than or equal support" in the REST test suite.
systemtests/rest_support.go Enhanced RestTestCase struct with new field ExpCodeGTE. Modified RunRestQueries to handle ExpCode and ExpCodeGTE. Renamed TestRestQueryIgnoreNumbers to RunRestQueriesIgnoreNumbers and updated its implementation.
tests/systemtests/authz_test.go Refactored test cases for authorization, improved error handling, updated expected outputs, and introduced helper functions for better maintainability.
tests/systemtests/bank_test.go Enhanced test cases for bank transactions, structured test cases for various scenarios, and updated expected status codes for GRPC queries.
tests/systemtests/distribution_test.go Renamed fields in RestTestCase struct for clarity and updated method calls for executing REST queries.
tests/systemtests/go.mod Updated dependency version for cosmossdk.io/systemtests.

Possibly related issues

  • Make System Test V2 pass #22753: The changes in this PR may help address issues related to making system tests pass, particularly in the context of improved error handling.

Possibly related PRs

  • ci: actually enable v2 system test #21539: This PR modifies the CI configuration to enable system tests for version 2, which may relate to the changes in error handling in the main PR as both involve testing and error management.
  • test: migrate e2e/bank to system tests #21607: This PR introduces system tests for the bank module, which may relate to the error handling improvements in the main PR, as both involve ensuring correct functionality and error reporting.
  • test(e2e/accounts): fix build #21725: This PR focuses on fixing build issues in the e2e tests, which may connect to the error handling improvements in the main PR, as both aim to enhance the reliability of the testing framework.
  • feat(bank/v2): Add MsgSend handler #21736: This PR adds a MsgSend handler and related tests, which could relate to the error handling in the main PR, as both involve ensuring proper functionality and error reporting in transactions.
  • test: migrate e2e/authz to system tests #21819: This PR migrates e2e tests to system tests, which may relate to the main PR's focus on improving error handling, as both aim to enhance the testing framework.
  • test: migrate e2e/distribution to system tests #21908: This PR migrates e2e tests for distribution to system tests, which may connect to the main PR's focus on error handling improvements, as both involve ensuring correct functionality.
  • test(systemtests): fix failing tests #22145: This PR addresses flaky integration tests, which may relate to the main PR's focus on error handling, as both aim to improve the reliability of the testing framework.
  • test: migrate e2e/mint to system tests #22294: This PR migrates e2e tests for minting to system tests, which may relate to the main PR's focus on error handling improvements, as both involve ensuring correct functionality.
  • test: x/accounts systemtests #22320: This PR introduces system tests for the x/accounts module, which may connect to the main PR's focus on error handling, as both aim to enhance the testing framework.
  • fix(server/v2/cometbft): proper query error #22746: This PR improves query error handling in the server, which directly relates to the main PR's focus on enhancing error reporting in the Query method.
  • ci: run runtime/v2 tests in CI #22769: This PR adds a CI job for runtime/v2 tests, which may relate to the main PR's focus on improving error handling, as both aim to enhance the testing framework.

Suggested labels

C:x/bank

Suggested reviewers

  • sontrinh16
  • akhilkumarpilli
  • tac0turtle

📜 Recent review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 9e94ea8 and 0dee687.

⛔ Files ignored due to path filters (1)
  • tests/systemtests/go.sum is excluded by !**/*.sum
📒 Files selected for processing (1)
  • tests/systemtests/go.mod (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/systemtests/go.mod

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Comment on lines +111 to +113
for key, value := range headers {
req.Header.Set(key, value)
}

Check warning

Code scanning / CodeQL

Iteration over map Warning

Iteration over map may be a possible source of non-determinism
@julienrbrt julienrbrt added the backport/v0.52.x PR scheduled for inclusion in the v0.52's next stable release label Dec 5, 2024
@github-actions github-actions bot added C:server/v2 Issues related to server/v2 C:server/v2 appmanager labels Dec 5, 2024
@julienrbrt julienrbrt marked this pull request as ready for review December 5, 2024 14:37
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (6)
systemtests/rest_support.go (1)

14-14: Use 'URL' instead of 'Url' for consistent naming

According to Go naming conventions, acronyms should be capitalized. Consider renaming the Url field to URL for consistency.

-	Url        string
+	URL        string
tests/systemtests/authz_test.go (1)

729-756: Consider improving test case readability.

While the test cases provide good coverage of pagination functionality, consider extracting the expected JSON responses into constants or helper functions to improve readability and maintainability.

Example refactor:

+const (
+    singleGrantResponse = `{"grants":[{"authorization":...}],"pagination":{"next_key":null,"total":"1"}}`
+    multipleGrantsResponse = `{"grants":[{"authorization":...},{"authorization":...}],"pagination":{"next_key":null,"total":"2"}}`
+)
 grantsTestCases := []systest.RestTestCase{
     {
         Name:    "expect single grant",
         Url:     fmt.Sprintf(grantsURL, granterAddr, grantee1Addr),
         ExpCode: http.StatusOK,
-        ExpOut:  fmt.Sprintf(`{"grants":[{%s}],"pagination":{"next_key":null,"total":"1"}}`, grant1),
+        ExpOut:  singleGrantResponse,
     },
🧰 Tools
🪛 Gitleaks (8.21.2)

744-744: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

tests/systemtests/distribution_test.go (2)

132-135: Consider adding error test cases for the params endpoint.

The current test only covers the success scenario. Consider adding test cases for:

  • Invalid block height
  • Network errors
  • Malformed requests

146-149: Add validation for malformed validator addresses.

Consider adding test cases for:

  • Malformed validator addresses
  • Non-existent validator addresses
  • Invalid address format

Also applies to: 159-162

tests/systemtests/bank_test.go (2)

283-283: Consider using exact status code matching where appropriate.

While GetRequestWithHeadersGreaterThanOrEqual is useful for v1/v2 compatibility, some test cases (like line 268's BadRequest) should expect exact status codes. Consider using conditional logic based on the API version being tested.


319-334: Enhance balance endpoint test coverage.

Consider adding test cases for:

  • Accounts with zero balance
  • Very large balance numbers
  • Multiple denominations in a single query
  • Pagination with large sets of denominations
📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 00c7756 and 9e94ea8.

⛔ Files ignored due to path filters (1)
  • tests/systemtests/go.sum is excluded by !**/*.sum
📒 Files selected for processing (7)
  • server/v2/appmanager/appmanager.go (1 hunks)
  • systemtests/CHANGELOG.md (1 hunks)
  • systemtests/rest_support.go (3 hunks)
  • tests/systemtests/authz_test.go (2 hunks)
  • tests/systemtests/bank_test.go (4 hunks)
  • tests/systemtests/distribution_test.go (4 hunks)
  • tests/systemtests/go.mod (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • systemtests/CHANGELOG.md
🧰 Additional context used
📓 Path-based instructions (6)
tests/systemtests/go.mod (1)

Pattern tests/**/*: "Assess the integration and e2e test code assessing sufficient code coverage for the changes associated in the pull request"

server/v2/appmanager/appmanager.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tests/systemtests/distribution_test.go (3)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern tests/**/*: "Assess the integration and e2e test code assessing sufficient code coverage for the changes associated in the pull request"


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

tests/systemtests/authz_test.go (3)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern tests/**/*: "Assess the integration and e2e test code assessing sufficient code coverage for the changes associated in the pull request"


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

tests/systemtests/bank_test.go (3)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern tests/**/*: "Assess the integration and e2e test code assessing sufficient code coverage for the changes associated in the pull request"


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

systemtests/rest_support.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

🪛 Gitleaks (8.21.2)
tests/systemtests/authz_test.go

744-744: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

🔇 Additional comments (7)
systemtests/rest_support.go (1)

131-133: Avoid non-determinism in map iteration over HTTP headers

Iteration over a map in Go can result in non-deterministic ordering. To ensure consistent behavior when setting headers, consider sorting the map keys before iteration.

server/v2/appmanager/appmanager.go (1)

233-233: Improved error wrapping enhances clarity

Wrapping the error with additional context in fmt.Errorf("invalid height: %w", err) provides clearer error messages and aids in debugging.

tests/systemtests/authz_test.go (3)

685-718: LGTM! Well-structured test cases for grant query endpoint.

The test cases provide good coverage of edge cases including invalid addresses, empty addresses, and invalid message types. The expected outputs are well-defined and match the expected behavior.


771-786: LGTM! Comprehensive test cases for grants by granter endpoint.

The test cases effectively cover error scenarios and successful queries, with clear expected outputs for each case.


798-813: LGTM! Well-structured test cases for grants by grantee endpoint.

The test cases provide good coverage of error scenarios and successful queries, maintaining consistency with other test sections.

tests/systemtests/distribution_test.go (1)

186-201: LGTM! Comprehensive test coverage for slashes endpoint.

The test cases effectively cover:

  • Invalid start height
  • Invalid end height
  • Valid request scenario
tests/systemtests/bank_test.go (1)

268-268: LGTM! Status code change aligns with v2 error handling.

The change from StatusInternalServerError to StatusBadRequest correctly implements the improved error handling in v2, where user errors return appropriate 4xx status codes instead of 500.

systemtests/rest_support.go Show resolved Hide resolved
@@ -2,6 +2,8 @@ module github.com/cosmos/cosmos-sdk/tests/systemtests

go 1.23

replace cosmossdk.io/systemtests => ../../systemtests
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Avoid committing local 'replace' directives in go.mod

The replace directive with a local path (../../systemtests) can cause build issues for others who do not have the same directory structure. It's recommended to remove local replacements before committing or use versioned modules.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will replace by a commit from here yeah

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@julienrbrt, great! Thank you for addressing this.

Copy link
Member

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love this, nice job

Copy link
Contributor

@hieuvubk hieuvubk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@julienrbrt julienrbrt added this pull request to the merge queue Dec 6, 2024
Merged via the queue into main with commit 4caac04 Dec 6, 2024
76 of 77 checks passed
@julienrbrt julienrbrt deleted the julien/test-system branch December 6, 2024 11:12
mergify bot pushed a commit that referenced this pull request Dec 6, 2024
(cherry picked from commit 4caac04)

# Conflicts:
#	server/v2/appmanager/appmanager.go
#	systemtests/CHANGELOG.md
#	systemtests/rest_support.go
#	tests/systemtests/go.mod
#	tests/systemtests/go.sum
julienrbrt added a commit that referenced this pull request Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/v0.52.x PR scheduled for inclusion in the v0.52's next stable release C:server/v2 appmanager C:server/v2 Issues related to server/v2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants