CRAYSAT-1939: add testing for sat version #649
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add the first functional test for the System Admin Toolkit (SAT). This is a proof of concept for future tests for SAT. This test simply executes the `sat --version` command on the NCN and verifies that the output matches the value in the file used to set the version. SAT tests are implemented in Python using the `unittest` standard library package. This provides an easy way to structure tests, add set up and tear down logic for each test, and run the tests from a script. The Python code for the SAT tests is all under a `sat_functional` subpackage, and individual subpackages and modules beneath that subpackage will implement tests for specific SAT functionality. Tests will be organized by the `sat` command they are testing. The `__main__.py` in the `sat_testing` module simply passes along the arguments, which should be a list of relative test paths, to the `unittest.TextTestRunner`, which then runs the tests. SAT tests are added in Gossfiles. Currently a single test file for SAT tests, `goss-sat-version.yaml`, is created. A test suite is created in the goss file `ncn-sat-functional-tests.yaml`. This test suite is limited to run on only `ncn-m001`. It will eventually include all SAT functional tests. Test Description: Downloaded the RPM built on Jenkins to drax. Extracted the RPM using `rpm2cpio $RPM_FILE | cpio -idvmu`. Modified the path to the Python virtual environment in the `test_sat_functional` script in the `bin` directory of the virtualenv to match the extraction path, set `GOSS_BASE` to the extracted Goss base directory, and then ran the Goss test file and test suite using `goss -g $GOSS_FILE validate`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary and Scope
Summarize what has changed. Explain why this PR is necessary. What is impacted? Is this a new feature, critical bug fix, etc?
Is this change backwards incompatible, backwards compatible, or a backwards compatible bugfix?
Issues and Related PRs
List and characterize relationship to Jira/Github issues and other pull requests. Be sure to list dependencies.
<insert branch name here><insert PR URL here>Testing
List the environments in which these changes were tested.
Tested on:
<development system>Test description:
How were the changes tested and success verified? If schema changes were part of this change, how were those handled in your upgrade/downgrade testing?
Risks and Mitigations
Are there known issues with these changes? Any other special considerations?
Pull Request Checklist