Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle ssl only scylla cluster setup #4114

Merged
merged 14 commits into from
Nov 22, 2024
Merged

fix: handle ssl only scylla cluster setup #4114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
2572685
fix: adds SSL_ENABLED flag to start scylla cluster in ssl only mode
VAveryanov8 Nov 15, 2024
7b7e591
fix: handle ssl only scylla clusters
VAveryanov8 Nov 15, 2024
9d47e77
fix: CQLAddr provides ssl or non-ssl addr depending on cluster conf.
VAveryanov8 Nov 19, 2024
735d689
fix(Makefile): use yq to produce scylla config with ssl enabled
VAveryanov8 Nov 19, 2024
f821fcc
fix: typo in testing/scylla/config/scylla-ssl.yaml
VAveryanov8 Nov 20, 2024
461ef99
fix(test): use scylla cluster with SSL for integration tests
VAveryanov8 Nov 20, 2024
62eb540
fix(cluster): simplifies SingleHostSessionOption when dealing with SSL
VAveryanov8 Nov 20, 2024
87cdcc0
fix(test): adds ssl support to cqlping integration tests
VAveryanov8 Nov 21, 2024
a3124d8
fix(test): adds ssl support to repair integration test
VAveryanov8 Nov 21, 2024
22412df
fix(test): adds ssl support to healthcheck integration tests
VAveryanov8 Nov 21, 2024
6895803
fix(test): unifies how SSL_ENABLED is used in testconfig
VAveryanov8 Nov 21, 2024
53889e0
fix(ci): adds missing ssl-enabled option for a one entry in ci config
VAveryanov8 Nov 21, 2024
83690f9
refactor: moves parsing of SSL_ENABLED env var to the testconfig pkg
VAveryanov8 Nov 21, 2024
63edf64
fix(test): use cqlping with ssl for the restore test of old scylla ver
VAveryanov8 Nov 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .github/actions/test-setup/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ inputs:
description: "Should this action run 'make start-dev-env'"
required: false
default: 'true'
ssl-enabled:
description: "Specifies if Scylla cluster should use ssl only configuration or not"
required: false
default: 'true'

runs:
using: "composite"
Expand All @@ -42,5 +46,5 @@ runs:

- name: Start dev env
if: inputs.start-dev-env == 'true'
run: make start-dev-env SCYLLA_VERSION=${{ inputs.scylla-version }} IP_FAMILY=${{ inputs.ip-family }} RAFT_SCHEMA=${{ inputs.raft-schema }} TABLETS=${{ inputs.tablets }}
shell: bash
run: make start-dev-env SCYLLA_VERSION=${{ inputs.scylla-version }} IP_FAMILY=${{ inputs.ip-family }} RAFT_SCHEMA=${{ inputs.raft-schema }} TABLETS=${{ inputs.tablets }} SSL_ENABLED=${{ inputs.ssl-enabled }}
shell: bash
12 changes: 11 additions & 1 deletion .github/cfg/integration-test-cfg.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,48 +2,58 @@
ip-family: IPV4
raft-schema: disabled
tablets: none
ssl-enabled: true

- scylla-version: scylla-enterprise:2023.1.11
ip-family: IPV4
raft-schema: enabled
tablets: none
ssl-enabled: true

- scylla-version: scylla-enterprise:2023.1.11
ip-family: IPV6
raft-schema: enabled
tablets: none
ssl-enabled: true

- scylla-version: scylla-enterprise:2024.1.12
ip-family: IPV4
raft-schema: none
tablets: none
ssl-enabled: true

- scylla-version: scylla-enterprise:2024.1.12
ip-family: IPV6
raft-schema: none
tablets: none
ssl-enabled: true

- scylla-version: scylla:6.2.0
ip-family: IPV4
raft-schema: none
tablets: disabled
ssl-enabled: true

- scylla-version: scylla:6.2.0
ip-family: IPV4
raft-schema: none
tablets: enabled
ssl-enabled: true

- scylla-version: scylla:6.2.0
ip-family: IPV6
raft-schema: none
tablets: enabled
ssl-enabled: false

- scylla-version: scylla-enterprise-nightly:latest-enterprise
ip-family: IPV4
raft-schema: none
tablets: disabled
ssl-enabled: true

- scylla-version: scylla-enterprise-nightly:latest-enterprise
ip-family: IPV4
raft-schema: none
tablets: enabled
tablets: enabled
ssl-enabled: false
19 changes: 12 additions & 7 deletions .github/cfg/integration-test-core.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
tablets: ${{ env.tablets }}
ssl-enabled: ${{ env.ssl-enabled }}

- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/restore RUN='"TestRestoreTables.*Integration"'
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/restore RUN='"TestRestoreTables.*Integration"'

restore-schema:
name: Test restore schema
Expand All @@ -44,11 +45,11 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
tablets: ${{ env.tablets }}

ssl-enabled: ${{ env.ssl-enabled }}
# Go does not support negative lookahead in regex expressions, so it has to be done manually.
# This regex ensures that all restore tests that didn't match restore-tables job will be run here.
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/restore RUN='"TestRestore([^T]|.{1}[^a]|.{2}[^b]|.{3}[^l]|.{4}[^e]|.{5}[^s]).*Integration"'
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/restore RUN='"TestRestore([^T]|.{1}[^a]|.{2}[^b]|.{3}[^l]|.{4}[^e]|.{5}[^s]).*Integration"'

backup:
name: Test backup
Expand All @@ -64,9 +65,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
tablets: ${{ env.tablets }}
ssl-enabled: ${{ env.ssl-enabled }}

- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/backup
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/backup

repair:
name: Test repair
Expand All @@ -82,9 +84,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
tablets: ${{ env.tablets }}
ssl-enabled: ${{ env.ssl-enabled }}

- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/repair
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/repair

small-pkg:
name: Test other, smaller packages
Expand All @@ -93,9 +96,10 @@ jobs:
- name: Check out code into the Go module directory
uses: actions/checkout@v3

- name: Set IP_FAMILY var for all tests
- name: Set IP_FAMILY and SSL_ENABLED var for all tests
run: |
echo "IP_FAMILY=${{ env.ip-family }}" >> $GITHUB_ENV
echo "SSL_ENABLED=${{ env.ssl-enabled }}" >> $GITHUB_ENV

- name: Setup testing dependencies
uses: ./.github/actions/test-setup
Expand All @@ -104,6 +108,7 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
tablets: ${{ env.tablets }}
ssl-enabled: ${{ env.ssl-enabled }}

- name: Run cqlping tests
run: make pkg-integration-test PKG=./pkg/ping/cqlping
Expand All @@ -127,4 +132,4 @@ jobs:
run: make pkg-integration-test PKG=./pkg/store

- name: Run migrate tests
run: make pkg-integration-test PKG=./pkg/schema/migrate
run: make pkg-integration-test PKG=./pkg/schema/migrate
5 changes: 5 additions & 0 deletions .github/cfg/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ type integrationTestCfg struct {
IPFamily string `yaml:"ip-family"`
RaftSchema string `yaml:"raft-schema"`
Tablets string `yaml:"tablets"`
SSLEnabled string `yaml:"ssl-enabled,omitempty"`
}

func (cfg integrationTestCfg) name() string {
Expand All @@ -29,6 +30,10 @@ func (cfg integrationTestCfg) name() string {
if cfg.Tablets == "enabled" {
parts = append(parts, "tablets")
}
if cfg.SSLEnabled == "false" {
parts = append(parts, "nossl")

}
return strings.Join(parts, "-")
}

Expand Down
17 changes: 12 additions & 5 deletions .github/workflows/integration-tests-2023.1.11-IPV4-raftschema.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ env:
ip-family: IPV4
raft-schema: enabled
tablets: none
ssl-enabled: "true"
jobs:
backup:
name: Test backup
Expand All @@ -19,9 +20,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/backup
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/backup
repair:
name: Test repair
runs-on: ubuntu-latest
Expand All @@ -34,9 +36,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/repair
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/repair
restore-schema:
name: Test restore schema
runs-on: ubuntu-latest
Expand All @@ -49,9 +52,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/restore RUN='"TestRestore([^T]|.{1}[^a]|.{2}[^b]|.{3}[^l]|.{4}[^e]|.{5}[^s]).*Integration"'
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/restore RUN='"TestRestore([^T]|.{1}[^a]|.{2}[^b]|.{3}[^l]|.{4}[^e]|.{5}[^s]).*Integration"'
restore-tables:
name: Test restore tables
runs-on: ubuntu-latest
Expand All @@ -64,24 +68,27 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/restore RUN='"TestRestoreTables.*Integration"'
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/restore RUN='"TestRestoreTables.*Integration"'
small-pkg:
name: Test other, smaller packages
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v3
- name: Set IP_FAMILY var for all tests
- name: Set IP_FAMILY and SSL_ENABLED var for all tests
run: |
echo "IP_FAMILY=${{ env.ip-family }}" >> $GITHUB_ENV
echo "SSL_ENABLED=${{ env.ssl-enabled }}" >> $GITHUB_ENV
- name: Setup testing dependencies
uses: ./.github/actions/test-setup
with:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run cqlping tests
run: make pkg-integration-test PKG=./pkg/ping/cqlping
Expand Down
17 changes: 12 additions & 5 deletions .github/workflows/integration-tests-2023.1.11-IPV4.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ env:
ip-family: IPV4
raft-schema: disabled
tablets: none
ssl-enabled: "true"
jobs:
backup:
name: Test backup
Expand All @@ -19,9 +20,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/backup
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/backup
repair:
name: Test repair
runs-on: ubuntu-latest
Expand All @@ -34,9 +36,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/repair
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/repair
restore-schema:
name: Test restore schema
runs-on: ubuntu-latest
Expand All @@ -49,9 +52,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/restore RUN='"TestRestore([^T]|.{1}[^a]|.{2}[^b]|.{3}[^l]|.{4}[^e]|.{5}[^s]).*Integration"'
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/restore RUN='"TestRestore([^T]|.{1}[^a]|.{2}[^b]|.{3}[^l]|.{4}[^e]|.{5}[^s]).*Integration"'
restore-tables:
name: Test restore tables
runs-on: ubuntu-latest
Expand All @@ -64,24 +68,27 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/restore RUN='"TestRestoreTables.*Integration"'
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/restore RUN='"TestRestoreTables.*Integration"'
small-pkg:
name: Test other, smaller packages
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v3
- name: Set IP_FAMILY var for all tests
- name: Set IP_FAMILY and SSL_ENABLED var for all tests
run: |
echo "IP_FAMILY=${{ env.ip-family }}" >> $GITHUB_ENV
echo "SSL_ENABLED=${{ env.ssl-enabled }}" >> $GITHUB_ENV
- name: Setup testing dependencies
uses: ./.github/actions/test-setup
with:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run cqlping tests
run: make pkg-integration-test PKG=./pkg/ping/cqlping
Expand Down
17 changes: 12 additions & 5 deletions .github/workflows/integration-tests-2023.1.11-IPV6-raftschema.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ env:
ip-family: IPV6
raft-schema: enabled
tablets: none
ssl-enabled: "true"
jobs:
backup:
name: Test backup
Expand All @@ -19,9 +20,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/backup
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/backup
repair:
name: Test repair
runs-on: ubuntu-latest
Expand All @@ -34,9 +36,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/repair
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/repair
restore-schema:
name: Test restore schema
runs-on: ubuntu-latest
Expand All @@ -49,9 +52,10 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/restore RUN='"TestRestore([^T]|.{1}[^a]|.{2}[^b]|.{3}[^l]|.{4}[^e]|.{5}[^s]).*Integration"'
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/restore RUN='"TestRestore([^T]|.{1}[^a]|.{2}[^b]|.{3}[^l]|.{4}[^e]|.{5}[^s]).*Integration"'
restore-tables:
name: Test restore tables
runs-on: ubuntu-latest
Expand All @@ -64,24 +68,27 @@ jobs:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run tests
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} PKG=./pkg/service/restore RUN='"TestRestoreTables.*Integration"'
run: make pkg-integration-test IP_FAMILY=${{ env.ip-family }} SSL_ENABLED=${{ env.ssl-enabled}} PKG=./pkg/service/restore RUN='"TestRestoreTables.*Integration"'
small-pkg:
name: Test other, smaller packages
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v3
- name: Set IP_FAMILY var for all tests
- name: Set IP_FAMILY and SSL_ENABLED var for all tests
run: |
echo "IP_FAMILY=${{ env.ip-family }}" >> $GITHUB_ENV
echo "SSL_ENABLED=${{ env.ssl-enabled }}" >> $GITHUB_ENV
- name: Setup testing dependencies
uses: ./.github/actions/test-setup
with:
ip-family: ${{ env.ip-family }}
raft-schema: ${{ env.raft-schema }}
scylla-version: ${{ env.scylla-version }}
ssl-enabled: ${{ env.ssl-enabled }}
tablets: ${{ env.tablets }}
- name: Run cqlping tests
run: make pkg-integration-test PKG=./pkg/ping/cqlping
Expand Down
Loading
Loading