Add test workflows based on salt-test (#616)

* Add unit test workflow based on salt-test

* Add integration test workflow based on salt-test

* Align boto imports in tests with Salt modules

Some modules `import boto` to set log levels. The related tests don't
import `boto`. This can cause a problem when `boto` is not available.

Tests are skipped when HAS_BOTO in the test_boto_*.py is False. Not
trying to `import boto` can leave HAS_BOTO=True in the test file even
though HAS_BOTO is False on the application side. In this case, tests
are not skipped and fail.

* Fix mock order in test_dig (test_network.py)

`salt.utils.path.which` should not be mocked before `network.__utils__`. The
Salt loader calls `salt.utils.network.linux_interfaces`, which needs the real
`salt.utils.path.which`.

* Fix mock calls

Signed-off-by: Pedro Algarvio <[email protected]>
(cherry picked from commit 3506e7f)

* Skip venafiapi test if vcert module not available

The same HAS_VCERT check is done in the runner module.

* Moving tests/integration/modules/test_cmdmod.py to pytest, Gareth J Greenaway original author

(cherry picked from commit 2c1040b)

* The repo.saltproject.io `index.html` file changed it's contents. Fix tests.

Signed-off-by: Pedro Algarvio <[email protected]>
(cherry picked from commit 672f658)

* Skip hwclock test when executed inside a container

* Skip git pillar tests when executed inside a container

These tests require a Git repository container, which is hard to correctly set
up when executing the tests in the container in GH Actions.

Using --network host can help, but there was still an error (the git repos were
empty) when I tried to set this up.

* Skip test requiring systemd inside container

* Exclude tests for hgfs if missing hglib

* Adjust GH actions to run all tests subsets and include functional

* Add new GH action to trigger scenarios tests

* Skip and fix tests when running on containers

* Fix some failing test causing problem in SUSE environments

* Skip more tests when running on containers

* Use skipif instead of skip_if as it seems not behaving equally

* Skip more tests that cannot run in a container

* Remove SSH test which doesn't make sense after lock mechanism

* Fix failing boto tests

* Skip unmaintained tests upstream around zcbuildout

* Arrange GH actions into fast and full executions

* GH Actions: Fix Salt version in _version.txt file

When GH Actions are executed, they skip the default entrypoint for the
container, so we need to arrange the _version.txt file manually

* Skip some tests that does not run well in GH Actions

Propagate GITHUB_ACTIONS variable to container environment

---------

Co-authored-by: Pedro Algarvio <[email protected]>
Co-authored-by: Gareth J. Greenaway <[email protected]>
Co-authored-by: Pablo Suárez Hernández <[email protected]>

.github/workflows/functional-fast-opensuse-leap.yml

@@ -0,0 +1,26 @@
+name: Fast functional tests
+
+on:
+  push:
+    branches: [openSUSE/release/3006.0]
+  pull_request:
+    branches: [openSUSE/release/3006.0]
+  workflow_dispatch:
+
+jobs:
+  fast-functional-tests:
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_ACTIONS: 1
+    container:
+      image: registry.opensuse.org/systemsmanagement/saltstack/products/testing/containers/leap15.5/containers/salt-testsuite-github:leap15.5
+      options: --hostname=salt-test-container
+    steps:
+      - name: Checkout Salt
+        uses: actions/checkout@v4
+      - name: Fix the Salt version in _version.txt file
+        run: rpm -q python3-salt-testsuite --queryformat '%{VERSION}' > $GITHUB_WORKSPACE/salt/_version.txt
+      - name: Run Fast functional tests
+        run: |
+          salt-test --skiplist https://raw.githubusercontent.com/openSUSE/salt-test-skiplist/main/skipped_tests.toml \
+            --directory . --package-flavor classic functional -- -vvv

.github/workflows/functional-full-opensuse-leap.yml

@@ -0,0 +1,26 @@
+name: Full functional tests
+
+on:
+  push:
+    branches: [openSUSE/release/3006.0]
+  pull_request:
+    branches: [openSUSE/release/3006.0]
+  workflow_dispatch:
+
+jobs:
+  full-functional-tests:
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_ACTIONS: 1
+    container:
+      image: registry.opensuse.org/systemsmanagement/saltstack/products/testing/containers/leap15.5/containers/salt-testsuite-github:leap15.5
+      options: --hostname=salt-test-container
+    steps:
+      - name: Checkout Salt
+        uses: actions/checkout@v4
+      - name: Fix the Salt version in _version.txt file
+        run: rpm -q python3-salt-testsuite --queryformat '%{VERSION}' > $GITHUB_WORKSPACE/salt/_version.txt
+      - name: Run Full functional tests
+        run: |
+          salt-test --skiplist https://raw.githubusercontent.com/openSUSE/salt-test-skiplist/main/skipped_tests.toml \
+            --directory . --package-flavor classic functional -- --core-tests --slow --run-expensive --ssh-tests --run-destructive -vvv

.github/workflows/integration-fast-opensuse-leap.yml

@@ -0,0 +1,26 @@
+name: Fast integration tests
+
+on:
+  push:
+    branches: [openSUSE/release/3006.0]
+  pull_request:
+    branches: [openSUSE/release/3006.0]
+  workflow_dispatch:
+
+jobs:
+  fast-integration-tests:
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_ACTIONS: 1
+    container:
+      image: registry.opensuse.org/systemsmanagement/saltstack/products/testing/containers/leap15.5/containers/salt-testsuite-github:leap15.5
+      options: --hostname=salt-test-container
+    steps:
+      - name: Checkout Salt
+        uses: actions/checkout@v4
+      - name: Fix the Salt version in _version.txt file
+        run: rpm -q python3-salt-testsuite --queryformat '%{VERSION}' > $GITHUB_WORKSPACE/salt/_version.txt
+      - name: Run Fast integration tests
+        run: |
+          salt-test --skiplist https://raw.githubusercontent.com/openSUSE/salt-test-skiplist/main/skipped_tests.toml \
+            --directory . --package-flavor classic integration -- -vvv

.github/workflows/integration-full-opensuse-leap.yml

@@ -0,0 +1,26 @@
+name: Full integration tests
+
+on:
+  push:
+    branches: [openSUSE/release/3006.0]
+  pull_request:
+    branches: [openSUSE/release/3006.0]
+  workflow_dispatch:
+
+jobs:
+  full-integration-tests:
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_ACTIONS: 1
+    container:
+      image: registry.opensuse.org/systemsmanagement/saltstack/products/testing/containers/leap15.5/containers/salt-testsuite-github:leap15.5
+      options: --hostname=salt-test-container
+    steps:
+      - name: Checkout Salt
+        uses: actions/checkout@v4
+      - name: Fix the Salt version in _version.txt file
+        run: rpm -q python3-salt-testsuite --queryformat '%{VERSION}' > $GITHUB_WORKSPACE/salt/_version.txt
+      - name: Run Full integration tests
+        run: |
+          salt-test --skiplist https://raw.githubusercontent.com/openSUSE/salt-test-skiplist/main/skipped_tests.toml \
+            --directory . --package-flavor classic integration -- --core-tests --slow --run-expensive --ssh-tests --run-destructive -vvv

.github/workflows/scenarios-fast-opensuse-leap.yml

@@ -0,0 +1,26 @@
+name: Fast scenarios tests
+
+on:
+  push:
+    branches: [openSUSE/release/3006.0]
+  pull_request:
+    branches: [openSUSE/release/3006.0]
+  workflow_dispatch:
+
+jobs:
+  fast-scenarios-tests:
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_ACTIONS: 1
+    container:
+      image: registry.opensuse.org/systemsmanagement/saltstack/products/testing/containers/leap15.5/containers/salt-testsuite-github:leap15.5
+      options: --hostname=salt-test-container
+    steps:
+      - name: Checkout Salt
+        uses: actions/checkout@v4
+      - name: Fix the Salt version in _version.txt file
+        run: rpm -q python3-salt-testsuite --queryformat '%{VERSION}' > $GITHUB_WORKSPACE/salt/_version.txt
+      - name: Run Fast scenarios tests
+        run: |
+          salt-test --skiplist https://raw.githubusercontent.com/openSUSE/salt-test-skiplist/main/skipped_tests.toml \
+            --directory . --package-flavor classic scenarios -- -vvv

.github/workflows/scenarios-full-opensuse-leap.yml

@@ -0,0 +1,26 @@
+name: Full scenarios tests
+
+on:
+  push:
+    branches: [openSUSE/release/3006.0]
+  pull_request:
+    branches: [openSUSE/release/3006.0]
+  workflow_dispatch:
+
+jobs:
+  full-scenarios-tests:
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_ACTIONS: 1
+    container:
+      image: registry.opensuse.org/systemsmanagement/saltstack/products/testing/containers/leap15.5/containers/salt-testsuite-github:leap15.5
+      options: --hostname=salt-test-container
+    steps:
+      - name: Checkout Salt
+        uses: actions/checkout@v4
+      - name: Fix the Salt version in _version.txt file
+        run: rpm -q python3-salt-testsuite --queryformat '%{VERSION}' > $GITHUB_WORKSPACE/salt/_version.txt
+      - name: Run Full scenarios tests
+        run: |
+          salt-test --skiplist https://raw.githubusercontent.com/openSUSE/salt-test-skiplist/main/skipped_tests.toml \
+            --directory . --package-flavor classic scenarios -- --core-tests --slow --run-expensive --ssh-tests --run-destructive -vvv

.github/workflows/unit-fast-opensuse-leap.yml

@@ -0,0 +1,26 @@
+name: Fast unit tests
+
+on:
+  push:
+    branches: [openSUSE/release/3006.0]
+  pull_request:
+    branches: [openSUSE/release/3006.0]
+  workflow_dispatch:
+
+jobs:
+  fast-unit-tests:
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_ACTIONS: 1
+    container:
+      image: registry.opensuse.org/systemsmanagement/saltstack/products/testing/containers/leap15.5/containers/salt-testsuite-github:leap15.5
+      options: --hostname=salt-test-container
+    steps:
+      - name: Checkout Salt
+        uses: actions/checkout@v4
+      - name: Fix the Salt version in _version.txt file
+        run: rpm -q python3-salt-testsuite --queryformat '%{VERSION}' > $GITHUB_WORKSPACE/salt/_version.txt
+      - name: Run Fast unit tests
+        run: |
+          salt-test --skiplist https://raw.githubusercontent.com/openSUSE/salt-test-skiplist/main/skipped_tests.toml \
+            --directory . --package-flavor classic unit -- -vvv

.github/workflows/unit-full-opensuse-leap.yml

@@ -0,0 +1,26 @@
+name: Full unit tests
+
+on:
+  push:
+    branches: [openSUSE/release/3006.0]
+  pull_request:
+    branches: [openSUSE/release/3006.0]
+  workflow_dispatch:
+
+jobs:
+  full-unit-tests:
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_ACTIONS: 1
+    container:
+      image: registry.opensuse.org/systemsmanagement/saltstack/products/testing/containers/leap15.5/containers/salt-testsuite-github:leap15.5
+      options: --hostname=salt-test-container
+    steps:
+      - name: Checkout Salt
+        uses: actions/checkout@v4
+      - name: Fix the Salt version in _version.txt file
+        run: rpm -q python3-salt-testsuite --queryformat '%{VERSION}' > $GITHUB_WORKSPACE/salt/_version.txt
+      - name: Run Full unit tests
+        run: |
+          salt-test --skiplist https://raw.githubusercontent.com/openSUSE/salt-test-skiplist/main/skipped_tests.toml \
+            --directory . --package-flavor classic unit -- --core-tests --slow --run-expensive --ssh-tests --run-destructive -vvv

tests/integration/externalapi/test_venafiapi.py

@@ -13,6 +13,14 @@
 from cryptography.hazmat.primitives import serialization
 from cryptography.x509.oid import NameOID
 
+try:
+    import vcert
+    from vcert.common import CertificateRequest
+
+    HAS_VCERT = True
+except ImportError:
+    HAS_VCERT = False
+
 from tests.support.case import ShellCase
 
 
@@ -36,6 +44,7 @@ def wrapper(self, *args, **kwargs):
     return wrapper
 
 
+@pytest.mark.skipif(HAS_VCERT is False, reason="The vcert module must be installed.")
 class VenafiTest(ShellCase):
     """
     Test the venafi runner
@@ -86,7 +95,6 @@ def test_request(self, name):
     @with_random_name
     @pytest.mark.slow_test
     def test_sign(self, name):
-
         csr_pem = """-----BEGIN CERTIFICATE REQUEST-----
 MIIFbDCCA1QCAQAwgbQxCzAJBgNVBAYTAlVTMQ0wCwYDVQQIDARVdGFoMRIwEAYD
 VQQHDAlTYWx0IExha2UxFDASBgNVBAoMC1ZlbmFmaSBJbmMuMRQwEgYDVQQLDAtJ